DTSRLOAD(1)

📦
version 2.5.3 · 2 versions
, in nixos 26.05

💖 dtsrload — like, the user cmd one 💖

okay bestie, gather round, because this is the page for dtsrload, a user command, and honestly she's kind of the main character of the DtSearch loading pipeline. Let's get into it 🫶

✨ NAME ✨

dtsrload — Load document objects in a database. that's it, that's the vibe. 👑

🎀 SYNOPSIS 🎀

dtsrload-ddbname  [-c]  [-tetxstr]  [-h0  | -hhashsz ]  [-ehufname]
[-pdotcnt] file
ngl the way the options nest right up against the program name with zero whitespace is a choice, but it is the choice, and we respect it. no spaces between an option and its value, ever. periodt. 💅

🌸 DESCRIPTION 🌸

so dtsrload does the whole "put the actual documents in the database" thing. it loads document header information and — in AusText type databases — the documents themselves into a DtSearch database. ✨

the input is a file of one or more documents in a simple canonical format, aka an fzk file. an fzk file can be generated by dtsrhan manually with a text editor, or by a special application program created for the purpose. typically the same fzk file is used for dtsrload and dtsrindex, but it is not required — and there are situations where it may not be desirable. (see dtsrfzkfiles(4) for the tea on DtSearch fzk files ☕)

dtsrload also maintains the current total document count in the database's configuration and status record. bookkeeping queen. 📊

now here's the key thing, and bestie you have to read this twice because the wording is wild: if a document's unique key in the fzk file does not preexist in the database, dtsrload considers the document to be new and does not add it as a new document. if the document's key already exists in the database, dtsrload totally replaces its record with the one in the fzk file. when duplicate record ids are encountered in a single fzk file, only the first occurrence of the document is loaded into the database, and the second one is discarded. duplicate record ids are maintained during execution with a hash table. 🗂️

dtsrload also performs a data compression function for documents that are actually stored in a database repository (that is, AusText type databases). in order to do this an encode compression huf file must be available. (see huffcode(1) for information about DtSearch document compression.)

and yeah, she says that exact same compression paragraph twice in the original. i'm keeping both because the rule is keep every detail and honestly? iconic of her. she really wanted you to know. 😭💖

dtsrload does not index the words used to access the database. that's dtsrindex's job. to prevent database link corruption, execute dtsrindex immediately after dtsrload. 🏃‍♀️💨

Caution: 🚨 To prevent database corruption, execute dtsrload only after all users of a preexisting database have exited their search programs — yes, all of them, no cap — to prevent database corruption. For a single fzk file, dtsrload must be executed immediately before dtsrindex so that dtsrindex can map the words it indexes to the correct internal database addresses. Only after both programs successfully complete execution may users again be allowed to perform online searches of the database. do not skip this step. do not pass go. 💀

💅 OPTIONS 💅

The following options are available:

Note: 📌 If an option takes a value, the value must be directly appended to the option name without white space. literally zero space. glued. attached. inseparable. 🫶
  • -ddbname — Specifies the 1 to 8 ASCII character name of the database to be updated. If an optional directory path is not prepended to the database name, dtsrload will attempt to open the database from the current working directory. File name extensions for database files are automatically appended — you don't gotta do a thing. ✨
  • -c — Instructs dtsrload to initialize the database total document count by counting existing records before loading the current batch. This option is usually not required. lowkey optional, highkey only if you know.
  • -tetxstr — Specifies the end of document text delimiter string. The default document separator in an fzk file is an ASCII form feed character (\f) followed by an ASCII line feed (\n). For certain multibyte languages it may be more convenient to specify a nonASCII string as the document delimiter. 🌈
  • -h0 — Instructs dtsrload to not check for duplicate record ids. This option should not be specified unless it is certain that there are no duplicate ids in the fzk file. ⚠️ like, certain certain.
  • -hhashsz — Sets the duplicate record id hash table size to hashsz. The default is 3000. dtsrload will execute more efficiently if the specified table size is larger than the number of documents in the fzk file. 📈
  • -ehufname — Sets the compression encode file name to hufname. The default is ophuf.huf. The file name can include a path prefix. This option is ignored unless the database type is AusText. 🙈
  • -pdotcount — Instructs dtsrload to print a progress character to stdout for every dotcount documents processed. The default is 20. little dots of encouragement 🫧

🦋 OPERANDS 🦋

The required input file name (file) identifies the file to be processed by dtsrload. It can optionally include a path prefix, either from root or relative to the current working directory. If a file name extension is not specified, dtsrload assumes a default extension of .fzk. cute of her to assume 🎀

🌟 ENVIRONMENT VARIABLES 🌟

None. nothing. empty. the void. 🕳️

🧚 RESOURCES 🧚

None. again, nothing. she's low maintenance. 💜

⭐ ACTIONS/MESSAGES ⭐

None. ✨

🔥 RETURN VALUES 🔥

The return values are as follows:

  • 0 — dtsrload completed successfully. slay. 🎉
  • 1 — dtsrload successfully recovered from an error. This occurs when one or more documents were discarded because of a partially invalid fzk file format, duplicate record ids, or empty record text. she's fine, she's fine, everything's fine 😮‍💨
  • >1 — dtsrload encountered a fatal error. 🪦

💖 FILES 💖

dtsrload reads the specified fzk file and opens all the database and related language files for the specified database name.

For AusText type databases, it also reads the compression encode file ophuf.huf.

dtsrload updates the following database files:

dbname.d00

dbname.d01

dbname.k00

dbname.k01

four files, all hers, all updated. 👑

🌸 EXAMPLES 🌸

Load database mydb with the documents specified in the fzk file named batch1.fzk in the current working directory:

dtsrload -dmydb batch1

Load database mydb with the documents specified in the fzk file /u/dtsearch/jpndocs.1. Three ASCII plus signs at the bottom of each document signals the end of document text and the beginning of the next fzk file record:

dtsrload -dmydb -t+++ /u/dtsearch/jpndocs.1

see how the +++ goes right up against the -t? no whitespace, bestie. we warned you. 💅

🌈 SEE ALSO 🌈

dtsrhan(1), dtsrindex(1), huffcode(1), dtsrfzkfiles(4), DtSearch(5)

go read them, they're all part of the friend group 🫶

okay that's the whole page, mwah 💋 this whole thing came off the dtsrload(user cmd) manual page, footer and all — that's the source/version line, and she didn't sneak a date in there either, so we're matching her energy exactly. now go load some documents, and remember: all users out first, dtsrindex immediately after, no spaces in your option values. iykyk. 🎀✨🦋

📦 where this page comes from