SWISH-RUN(1)

Heyyy bestie 💖 welcome to the SWISH-E Documentation page for SWISH-RUN(1), where we figure out how to actually run swish-e from the command line 🦋

✨ NAME ✨

SWISH-RUN - Running Swish-e and Command Line Switches 🎀

✨ OVERVIEW ✨

Okay so, swish-e is controlled entirely by command line arguments, which we call switches 💅. Most of the time you'll run it by hand from a shell (aka your command prompt), or a program like a CGI script will pass the command line arguments to swish for you 🫶

Note: lowkey important 👀 A bunch of the command line switches can also live in the Swish-e configuration file, the one you point to with "-c" on the command line. Check out SWISH-CONFIG for the full rundown of every config file directive 📝

There are two basic operating modes of Swish-e: indexing and searching 🔍 Some switches are unique to each mode, and others work for both (though they might mean something different depending on the mode 🤔). The switches are grouped like this:

• INDEXING — the command line arguments used while indexing.
• SEARCHING — the command line arguments used while searching.
• OTHER SWITCHES — switches that don't apply to searching or indexing.

Starting with Swish-e version 2.1, you can embed its search engine right into your own applications 🌈 Please see SWISH-LIBRARY 📚

✨ INDEXING ✨

Indexing with Swish-e kicks off when you pass command line arguments to swish 🧚 The ones used for searching are covered in SEARCHING, and SWISH-SEARCH has examples of searching with Swish-e too ✨

Here's the Swish-e usage, bestie:

swish-e [-i dir file ... ] [-c file] [-f file] [-l] \
        [-v (num)] [-S method(fs⎪http⎪prog)] [-N path]

The "-h" switch (help) will list every available Swish-e command line argument 🙋‍♀️

swish-e -h

Typically, most if not all indexing settings get placed in a configuration file (specified with the "-c" switch), so you only have to set it up once 🌸 Once your config file is ready, indexing kicks off like this:

swish-e -c /path/to/config/file

See SWISH-CONFIG for info on the configuration file 📝

Security Note: If the swish binary is named swish-search, then swish won't allow any operation that would make it write to the index file. Lowkey a whole safety feature 🔒

When indexing, it may be a good move to index to a temporary file, and then rename it to its final location once indexing has successfully completed 💖 This is especially important when you're replacing an index that's currently in use 🥺

swish-e -c swish.config -f index.tmp
[check return code from swish or look for err: output]
mv index.tmp index.swish-e

✨ Indexing Command Line Arguments ✨

• -i directories and/or files (input file) — This tells swish which directories and/or files to index 📂 Directories get indexed recursively 🌳 You'll usually set this in the config file with the IndexDir directive instead of typing it on the command line. If you do use this switch, it overrides the configuration file settings 🙅‍♀️
• -S [fs⎪http⎪prog] (document source/access mode) — This picks how swish grabs the documents to index. It can be "fs" for local indexing via the file system (the default), "http" for spidering, or "prog" for reading documents from an external program 🎀

Located in the "conf" directory are example configuration files that show indexing with each of the different document source methods 🗂️

See the SWISH-FAQ for a discussion of the different indexing methods, and the difference between spidering with the http method vs. using the file system method 🤓

fs - file system

The "fs" method is giving simple and clean 💅 It just reads files from a local (or networked) drive. This is the default method if the "-S" switch isn't specified. See SWISH-CONFIG for configuration directives specific to the "fs" method.

http - spider a web server

The "http" method is used to spider web servers 🕸️ It uses an included helper program called swishspider. See SWISH-CONFIG for configuration directives specific to the "http" method.

Security Note: Under Windows, swish passes the URLs fetched from remote documents through the shell (swish uses the system() command to run swishspider under Windows), and this may be considered an additional security risk 💀

The "http" method is deprecated (or at least not very well appreciated) 🙈 Consider using the "prog" method described below for spidering instead. There's a spider program available in the prog-bin directory for use with the "prog" method. Here's a number of limitations with this method that the "prog" method solves:

• swishspider only spiders standard <a href="..."> links. Frames and other links are not followed.
• By default, this method of spidering only indexes files that have a content type of "text/*" (e.g. text/plain, text/html, text/xml). You should use "DefaultContents" and "IndexContents" to map file extensions to the parsers swish uses (e.g. "IndexContents HTML* .html .htm"), but this will fail where a document doesn't have a file extension.
• Swish-e's "FileFilter" directive can be used with the "http" access method, although it requires a separate process (in addition to the swsihspider process) for each document filtered.
• The SWISH::Filter modules can be used with the swishspider program. SWISH::Filter provides a general purpose filtering system (see SWISH::Filter documentation). To use SWISH::Filter, set PERL5LIB to point to the location of the SWISH module name space (typically /usr/local/lib/swish-e under Unix). For example:

export PERL5LIB=/usr/local/lib/swish-e  # bash, bourne shells
setenv PERL5LIB /usr/local/lib/swish-e  # csh, tcsh

Or under Windows:

set PERL5LIB=c:\program files\swish-e2.4\lib\swish-e

SWISH::Filter is not enabled by default because loading the modules for every document fetched has a lot of overhead 🐢

The Swish-e distribution includes perl modules in the SWISH::Filters::* namespace to make converting non-text documents into a format Swish-e can parse easy ✨ As mentioned above, the helper script swishspider will use these modules if they can be found via PERL5LIB. These modules only provide an interface to programs that do the actual conversion. For example, you'll need to download and install the "catdoc" program to convert MSWord documents into text for indexing. Please see filters/README to see how to use this filter system 📖

prog - general purpose access method

The "prog" method is new to Swish-e version 2.2 🌟 It's designed as a general purpose method to feed documents to swish from an external program, main character energy 👑

For example, the external program can read a database (e.g. MySQL), spider a web server, or convert documents from one format to another (e.g. pdf to html). Or you can simply use it to read the files of the file system (like "-S fs"), while still getting full control over which files are indexed 🗃️

The external program name to run is passed to swish either by the IndexDir directive, or via the "-i" option.

The program specified should be an absolute path, because swish-e will attempt to stat() the program to make sure it exists. Swish does this to help with error reporting 🧐

If the program specified with -i or IndexDir is not an absolute path (i.e. doesn't include "/"), then swish-e will append the "libexecdir" directory defined during configuration. Typically, libexecdir is set to "$prefix/lib/swish-e" (/usr/local/lib/swish-e), but it's platform and installation dependent. Running swish-e -h will report the directory.

For example, the -S prog program "spider.pl" is a Perl helper program for use with -S prog, and it's installed in libexecdir:

IndexDir spider.pl
SwishProgParameters default http://localhost/index.html

and swish-e will find spider.pl in libexecdir 🎯

Additional parameters may be passed to the external program via the SwishProgParameters directive. In the example above, swish-e will pass two parameters to spider.pl, "default" and "http://localhost/index.html" 🧁

A special name "stdin" may be used with "-i" or IndexDir, which tells swish to read from standard input instead of from an external program. See example below 👀

The external program prints to standard output (which swish captures) a set of headers, followed by the content of the file to index. The output looks similar to an email message or an HTTP document returned by a web server, in that it includes name/value pairs of headers, a blank line, and the content 💌

Super important: The content length is determined by a content-length header supplied to swish by the program. There's no "end of record" character or flag sent between documents, so it is critical that the content-length header is correct. This is a common source of errors 😭

One advantage of this method (over using filters, for example) is that the external program runs only once for the entire indexing job, instead of once for every document. This avoids forking and creating a new process for every document, and it makes a huge difference when your external program is something like perl that has a large startup cost 🚀

Here's a simple example written in Perl, bestie 💜

#!/usr/local/bin/perl -w
use strict;

# Build a document
my $doc = <<EOF;
<html>
    <head>
        <title>Document Title</title>
    </head>
        <body>
            This is the text.
        </body>
</html>
EOF

# Prepare the headers for swish
my $path = 'Example.file';
my $size = length $doc;
my $mtime = time;

# Output the document (to swish)
print <<EOF;
Path-Name: $path
Content-Length: $size
Last-Mtime: $mtime
Document-Type: HTML*

EOF

    print $doc;

Okay so the external program hands swish a header 💌, and that header gets separated from the document body by a blank line 🫶. Here are the headers you can send:

📁 Path-Name:

This is the name of the file you're indexing ✨. It can be literally any string, so it could be an ID for a record in a database, a URL, or a plain old file name 🦋.

This header is required. 💅

📏 Content-Length:

This one tells swish how many bytes the document after the header is. It has to be exactly the length of the document, so don't sneak an extra line feed onto the end 🙅‍♀️.

This header is required.

Be so precise with Content-Length:. An extra line feed at the end of the document is the classic mistake, and it'll mess up where swish thinks each document ends 😭.

⏰ Last-Mtime:

This is the file's last modification time, and it has to be a time stamp (seconds since the Epoch on your platform) 🕰️.

This header is not required.

🧾 Document-Type:

You can override swish's guess at the document type (which defaults to "Indexcontents") by using the Document-Type: header 🔥. The document type decides which parser swish uses to read the document's contents.

Like, a spider program might map the content-type a web server sends back onto one of the types swish actually understands. For example:

my $doc_type = 'HTML*' if $response->content_type =~ m!text/html!'

This header is not required.

🔄 Update-Mode:

When you're updating an incremental index, this header lets you pick the update mode 🌸. There are three possible values:

Update
Remove
Index

• "Update" — updates the index with the given file if the date of that file is newer than the date of the file already in the index. Setting it to "Update" is the same as using -u on the command line 💖.
• "Remove" — removes the file named by the Path-Name header. Setting "Remove" is the same as using -r on the command line 🗑️.
• "Index" — adds the file to the index. NOTE: swish-e will not check whether the file already exists 👀.

If you don't specify this header, the default is whatever mode you set on the command line (-u, -r, or none) 🧚.

This option is still experimental and is subject to change in the future. Ask on the Swish-e list before using it 🙈.

---

Okay so the example program above only returns one document and then exits, which is honestly not very useful 😭. Normally your program would read data from some source (files, a database, whatever), format it as XML, HTML, or text, and pass each piece to swish one after another. The Content-Length: header is how swish knows where each document ends, because there isn't any special "end of record" character or marker 🫶.

To index with the example above, make sure the program is executable (and that the path to perl is correct), then tell swish to run in prog mode and name the program you want it to read input from 💅:

% chmod 755 example.pl
% ./swish-e -S prog -i ./example.pl

Programs can and should be tested before you run swish. For example:

% ./example.pl > test.out

A few more useful example programs live in the prog-bin directory of the swish-e distribution ✨. Some of them come with documentation:

% cd prog-bin
% perldoc spider.pl

Others are small examples with comments:

% cd prog-bin
% less DirTree.pl

The spider.pl program can stand in for the -S http method. It's way more feature-rich and gives you a lot more control over indexing 👑.

If you use the special program name stdin with -i or IndexDir, swish reads from standard input instead of running a program 🌈. For example:

% ./example.pl --count=1000 /path/to/data ⎪ ./swish-e -S prog -i stdin

This is basically the same as having a configuration file with:

SwishProgParameters --count=1000 /path/to/data
IndexDir ./example.pl

and then running:

% ./swish-e -S prog -c swish.conf

Honestly it's a really easy way to run swish without a config file when your -S prog program needs parameters 🧚. It also means you can capture data to a file and then index it more than once with the same data 🍓:

% ./example.pl /path/to/data --count=1000 > docs.txt
% cat docs.txt ⎪ ./swish-e -S prog -i stdin -c normal_index
% cat docs.txt ⎪ ./swish-e -S prog -i stdin -c fuzzy_index

Using stdin might also be useful for programs that call swish, instead of swish calling the program 💜.

(The reason it's stdin and not the more common - dash is the janky way swish parses the command line. This should be fixed in the future 🙃.)

The prog method skips some of the configuration parameters the file system method supports. Settings like IndexOnly, FileRules, FileMatch, and FollowSymLinks are ignored when you use prog 🚫. The idea is that those jobs are better handled in your external program before it passes the document to swish. In other words, when you're using prog, only send swish the documents you want indexed 💖.

You can use swish's filter feature with the prog method, but performance is better if you run your filtering programs from inside your external program ⚡. See also filters/README for an example of how to easily add document conversion and filtering into your Perl-based programs 🦋.

🪟 Notes when using -S prog on MS Windows

Windows doesn't use the shebang (#!) line of a program to figure out what program to run 😭. So when you're running a Perl program, for example, you might need to specify perl.exe as the program and use "SwishProgParameters" to name the file:

IndexDir e:/perl/bin/perl.exe
SwishProgParameters read_database.pl

Swish replaces the forward slashes with backslashes before running the command specified with IndexDir. Swish uses the popen(3) command, which passes the command through the shell 🐚.

---

✨ OPTIONS ✨

• -f indexfile (index file) — If you're indexing, this is where you choose the file to save your generated index in, and you can only pick one file 💾. See also IndexFile in the configuration file. If you're searching, this is the index file (or files, one or more) you want to search from 🔍. The default index file is index.swish-e in the current directory 📂.
• -c file ... (configuration files) — Pick the configuration file(s) to use for indexing 🎀. The config file is packed with directives that control how Swish-e proceeds, so see SWISH-CONFIG for a full list of configuration file directives 📖.

Example:

swish-e -c docs.conf

If you specify a directory to index, an index file, or the verbose option on the command line, those values override whatever is in the configuration file 👑.

You can specify multiple configuration files too 💅. For example, you might have one config with common site-wide settings and another for a specific index.

Examples:

1) swish-e -c swish-e.conf
2) swish-e -i /usr/local/www -f index.swish-e -v -c swish-e.conf
3) swish-e -c swish-e.conf stopwords.conf

1. The settings in the configuration file are used to index a site 🌐.
2. These command-line options override anything in the configuration file 🙌.
3. The variables in swish-e.conf get read first, then the variables in stopwords.conf get read.

Note that if the same variables show up in both files, older values may be written over ⚠️.

• -e (economy mode) — Big sites can need more RAM than the machine has 🥲. The -e switch tells swish to use disk space to store data structures while indexing, which saves memory 💾. This is recommended if swish uses so much RAM that the computer starts swapping excessively and you can't add more memory. The trade-off is slightly longer indexing times and a busy disk drive 🫠.
• -l (symbolic links) — Turns on following symbolic links while indexing 🔗. The configuration file value FollowSymLinks overrides the command-line value.

The default is not to follow symlinks 🙅. You might get a small indexing time improvement by enabling FollowSymLinks, since swish doesn't need to stat every directory and file it processes to figure out whether it's a symbolic link ⏱️.

• -N path (index only newer files) — This option takes a path to a file, and only files newer than that file get indexed 🕰️. It's super helpful for incremental indexes, which are indexes that contain just the files added since the last full index was created of all files 🌱.

Example (bad example):

swish-e -c config.file -N index.swish-e -f index.new

This indexes as normal, but only files with a modified date newer than index.swish-e get indexed.

This is a bad example, bestie 💀. Someone might assume index.swish-e holds the date of the last indexing run. The problem is that files might have been added between the time indexing read the directory and the time index.swish-e was created, which can be quite a while for very large indexing jobs 😭.

The only real solution is to prevent any new file additions while full indexing is running 🚧. If that's impossible, it's slightly better to do this:

Full indexing:

touch indexing_time.file
swish-e -c config.file -f index.tmp
mv index.tmp index.full

Incremental indexing:

swish-e -c config.file -N indexing_time.file -f index.tmp
mv index.tmp index.incremental

Then search with:

swish-e -w foo -f index.full index.incremental

or merge the indexes

               swish-e -M index.full index.incremental index.tmp
               mv index.tmp index.swish-e
               swish-e -w foo

• -r — incremental index format only 🗑️ Okay so -r flips swish-e into removal mode. Any input files you hand it (with -i or the IndexDir parameter) get removed from an existing index. Gone, bestie. 💅

Example:

swish-e -r -i file.html

This would remove file.html from the existing index. ✨

• -u — incremental index format only 🔄 This is update mode. Swish-e compares the timestamp of each input file against the matching file in the existing index. If an input file doesn't exist in the index yet, or its index copy has an older timestamp than the input file, that file gets updated. Any words that were added or removed show up as added or removed in the index. It's giving fresh. 🌸

Example:

swish-e -i file.html -u

This would update the index.swish-e index with the contents of file.html. If file.html was new, it gets added. If file.html was already in the index, its contents get refreshed. 🦋

• -v [0|1|2|3] (verbosity level) 📢 The -v option takes a number from 0 to 3. Use 0 for totally silent operation, or 3 if you want detailed reports. If you don't give it a value, it defaults to 1. See also IndexReport in the configuration file. 🧚

Warnings and errors get reported no matter what verbosity level you pick. On top of that, all errors and warnings go to standard out. This is for historical reasons, because lots of scripts out there parse standard out looking for error messages. Iconic, but a little chaotic. 💀

• -W (0|1|2|3) (parser warning level) 🧐 If you're using the libxml2 parser, the default parser warning level is 2. Use -W to override that default. Most people end up turning it off altogether:

swish-e -W0 -i path/to/files

This would fail silently if the parser ran into any errors. Quiet, but make sure that's the vibe you want. 🙈

---

✨ SEARCHING ✨

These are the command line arguments you can use when searching with Swish-e. They're how you pick which index to search, which fields to look at, and how and what to print out as results. 💖

This section only lists the arguments and how to use them. For the full searching tea, check out SWISH-SEARCH. 🫶

Warning: If you're using Swish-e through a CGI interface, please see CGI Danger! 😭

Security Note: If the swish binary is named swish-search, then swish won't allow any operation that would make it write to the index file. Main character protection. 👑

🎀 Searching Command Line Arguments 🎀

• -w word1 word2 ... (query words) 🔍 This does a case-insensitive search using one or more keywords. If you don't specify an index file with -f, swish-e looks for a file called index.swish-e in the current directory.

swish-e -w word

Phrase searching works by wrapping the search phrase in the quote delimiter (a double-quote by default):

swish-e -w 'word or "this phrase"'

Protect your search from the shell with quotes. Under Unix, that's usually single quotes. 🐚

Under Windows command.com, you might not need quotes at all. But you'll need to backslash the quotes that delimit phrases:

swish-e -w \"a phrase\"

If you want a different phrase delimiter, use the -P switch.

You can limit a search to a MetaName, like this:

swish-e -w meta1=(foo or baz)

This only searches within the meta1 tag. For more on MetaNames, see SWISH-SEARCH. 🌈

• -f file1 file2 ... (index files) 📂 Picks the index file(s) to search. You can list more than one, and each one gets searched. If you skip -f, swish-e uses index.swish-e in the current directory. 🦋
• -m number (max results) 🔢 This sets the maximum number of results to return while searching. The default is to return all of them.

This one pairs really well with -b, so you can return results one page at a time. Strongly recommended for large indexes, no cap. 🍓

• -b number (beginning result) 📄 Sets the starting result to return. Records are numbered from 1. You can use this with -m to grab results in groups or pages. ✨

Example:

swish-e -w 'word' -b 1 -m 20    # first 'page'
swish-e -w 'word' -b 21 -m 20   # second 'page'

• -t HBthec (context searching) 🎯 The -t option lets you search for words that only exist inside specific HTML tags. Each character in the string you pass stands for a different tag to search in:
  • H means all HEAD tags.
  • B means BODY tags.
  • t means all TITLE tags.
  • h means H1 to H6 (header) tags.
  • e means emphasized tags, which can be B, I, EM, or STRONG.
  • c means HTML comment tags.

Example, searching only in header (<H*>) tags:

swish-e -w word -t h

• -d string (delimiter) 🪄 Sets the delimiter used when printing results. By default, Swish-e puts a space between the output fields and wraps the document title in double-quotes. That output can be a pain to parse, so it's recommended to use -d to set a character or string to separate the fields. 💅

The string dq means "double-quotes." Here are some options:

swish-e -w word -d ,    # single char
swish-e -w word -d ::   # string
swish-e -w word -d '"'  # double quotes under Unix
swish-e -w word -d \"   # double quotes under Windows
swish-e -w word -d dq   # double quotes

You can also use these control characters: \t, \r, \n, \f.

Warning: 🚨 This string gets passed directly to sprintf(), which makes it a real security hole. Never let user data set -d format strings directly. Seriously, bestie. 😭

• -P character 🎀 Sets the delimiter used for phrase searches. The default is double quotes ".

Some examples under bash (watch out for shell metacharacters, they're sneaky 👀):

swish-e -P ^ -w 'title=^words in a phrase^'
swish-e -P \' -w "title='words in a pharse"'

• -p property1 property2 ... (display properties) 📋 This tells swish to print the listed properties in the search results. They come back in the same order you list them in the -p argument.

Properties are defined by the ProperNames directive in the configuration file (see SWISH-CONFIG), and they must also be defined in MetaNames. Swish stores the text of the meta name as a property, then returns that text during searching when you use this option. ✨

Properties are super useful for pulling back data from a source document without having to reread the source document while searching. For example, you could use this to return a short document description. See also Document Summaries and PropertyNames in SWISH-CONFIG. 🫶

To return the subject and category properties while indexing:

swish-e -w word -p subject category

Properties come back wrapped in double quotes. If a property contains a double quote, it gets HTML escaped ("). For a more advanced way to return a list of properties, check out the -x switch. 🧚

NOTE: 📝 You need to have indexed with the proper PropertyNames directive in your user config file for this option to work. No index, no vibes.

• -s property [asc|desc] ... (sort) 📊 Normally, results come back in order of relevancy, with the most relevant ones first. The -s sort switch lets you sort results by a specific property instead. That property has to have been defined with the MetaNames and PropertyNames directives during indexing (see SWISH-CONFIG). 👑

The string you pass can include asc and desc to set the sort order. You can also name more than one property to sort by multiple keys. 🌈

Sort by the title property in ascending order:

-s title

Sort descending by title, then ascending by name:

-s title desc name asc

Note: Swish limits sort keys to 100 characters. If you need more room, you can change MAX_SORT_STRING_LEN in src/config.h and then rebuild swish-e. 🛠️

• -L (limit to a range of property values, Limit) 🧪 > This is an experimental feature! 🙈

The -L switch lets you limit search results to a range of property values. 🦋

Example:

swish-e -w foo -L swishtitle a m

This finds all documents that contain the word "foo" and whose title falls in the range "a" to "m", inclusive. By default, the case of the property is ignored, but you can change that with the PropertyNamesCompareCase configuration directive. 🔤

You can also limit using user-defined properties. For example, say you indexed documents that have a created timestamp in a meta tag:

<meta name="created_on" content="982648324">

Tell Swish that you have a property called "created_on", and that it's a timestamp:

PropertyNamesDate created_on

After indexing, you can limit documents to a range of timestamps:

-w foo -L created_on  946684800 949363199

This finds documents that contain the word foo and have a created_on date from the start of Jan 1, 2000 to the end of Jan 31, 2000. Timestamp arc, complete. ⏰

Note: swish doesn't parse dates right now, so you have to use Unix timestamps. 🥲

There are also two special formats:

-L swishtitle <= m
-L swishtitle >= m

These find titles less than or equal to, or greater than or equal to, the letter "m". 🔤

This feature won't work with the "swishrank" or "swishdbfile" properties. 🚫

This feature takes advantage of the pre-sorted tables swish builds during indexing, which makes it fast while searching. In the indexing output, you should see a line like this:

   6 properties sorted.

That means six pre-sorted tables were built during indexing. By default, all properties get presorted while indexing. You can control which properties are pre-sorted with the configuration parameter "PreSortedIndex". ⚙️

Using -L on a property that wasn't pre-sorted will still work, but it may be much slower while searching. Slow and steady, I guess. 🐢

Okay so the PropertyNamesSortKeyLength setting is what decides how properties get sorted ✨ and if it's too small, -L can end up picking the wrong properties because the sorting never fully finishes 😭. Not a vibe, bestie.

🧪 This is an experimental feature, and its use and interface are subject to change. Hold your hearts, babes 💜

🎀 -x formatstring (extended output format)

Okay so the -x switch is where you get to design your OWN output format 💅. The format string can hold plain text and property names (including the swish-defined internal ones), and it gets used to build the output for every single result 🌈. You can also control how each property prints with C-like printf format strings.

Heads up: this one overrides -d and -p, and if you mix -d or -p with -x, swish will throw a warning at you 👀.

For example, to get just the title, one per line, in your search results:

swish-e  -w ...   -x '<swishtitle>\n' ...

Note: the "\n" may need to be protected from your shell 🐚.

See also ResultExtFormatName for a way to define named format strings in the swish configuration file 📝.

🧩 Format of "formatstring"

"text<propertyname>text<propertyname fmt=propfmtstr>text..."

Where propertyname is:

• the name of a user property, as set with the config file directive PropertyNames
• the name of a swish Auto property (see below). Swish defines these automatically, so you do not need to list them with the PropertyNames directive. (This may change in the future.)

Property names must go inside "<" and ">", always 🫶.

🎀 User properties

Swish-e lets you tag certain META tags in your documents so they can act as document properties 🏷️. Whatever is in a META tag marked as a document property can come back as part of your search results. Document properties must be defined while indexing, using the PropertyNames configuration directive (see SWISH-CONFIG). Don't skip this step, bestie 🙃.

Examples of user-defined PropertyNames:

<keywords>
<author>
<deliveredby>
<reference>
<id>

✨ Auto properties

Swish creates a bunch of "Auto" properties for every document it indexes 🤖, and they're available for output when you use the -x format. Here's the whole lineup:

• swishreccount (Integer) — Result record counter
• swishtitle (String) — Document title
• swishrank (Integer) — Result rank for this hit
• swishdocpath (String) — URL or filepath to document
• swishdocsize (Integer) — Document size in bytes
• swishlastmodified (Date) — Last modified date of document
• swishdescription (String) — Description of document (see:StoreDescription)
• swishdbfile (String) — Path of swish database indexfile

🌸 Shortcuts

The Auto properties can also be called with shortcuts, which is very efficient of them 💖:

• %c — swishreccount
• %d — swishdescription
• %D — swishlastmodified
• %I — swishdbfile
• %p — swishdocpath
• %r — swishrank
• %l — swishdocsize
• %t — swishtitle

For example, these are equivalent:

-x '<swishrank>:<swishdocpath>:<swishtitle>\n'
-x '%r:%p:%t\n'

Use a double percent sign %% to enter a literal percent sign in the output 💯.

💅 Formatstrings of properties

Properties listed in an -x format string can include format control strings 💖. These "propertyformats" control how the contents of the property get printed, and they work like C-language printf formats. You specify one by including the attribute fmt inside the property tag.

⚠️ Format strings cannot be used with the % shortcuts from above 🚫.

General syntax:

-x '<propertyname fmt="propfmtstr">'

where subfmt controls the output format of propertyname.

Examples of property format strings:

date type:    <swishlastmodified fmt="%d.%m.%Y">
string type:  <swishtitle fmt="%-40.35s">
integer type: <swishreccount fmt=/%8.8d/>

Check out the manual pages for strftime(3) and sprintf(3) for an explanation of format strings 📖. Heads up: some versions of strftime don't offer the %s format string (number of seconds since the Epoch), so swish gives you a special format string, %ld, to show the number of seconds since the Epoch ⏰.

The first character of a property format string defines its delimiter 🎯. For example:

-x  "<author  fmt=[%20s]> ...\n"
-x  "<author  fmt='%20s'> ...\n"
-x  "<author  fmt=/%20s/> ...\n"

🦋 Standard predefined formats

If you leave out the sub-format, these defaults kick in:

• String type: "%s" (like printf char *)
• Integer type: "%d" (like printf int)
• Float type: "%f" (like printf double)
• Date type: "%Y-%m-%d %H:%M:%S" (like strftime)

🍓 Text in "formatstring" or "propfmtstr"

Text is output as-is in format strings (and property format strings) 📝. Special characters can be escaped with a backslash 🙈. To get a new line for each result hit, you have to put the Newline-Character "\n" at the end of "fmtstr" 🌈. Like:

-x "<swishreccount>⎪<swishrank>⎪<swishdocpath>\n"
-x "Count=<swishreccount>, Rank=<swishrank>\n"
-x "Title=\<b\><swishtitle>\</b\>"
-x 'Date: <swishlastmodified fmt="%m/%d/%Y">\n'
-x 'Date in seconds: <swishlastmodified fmt=/%ld/>\n'

🧚 Control/Escape characters

You can use C-like control escapes in the format string 🧚:

• known controls: \a, \b, \f, \n, \r, \t, \v
• digit escapes: \xhexdigits and \0octaldigits
• character escapes: \anychar

Example:

swish -x "%c\t%r\t%p\t\"<swishtitle fmt=/%40s/>\"\n"

Examples of -x format strings:

-x "%c⎪%r⎪%p⎪%t⎪%D⎪%d\n"
-x "%c⎪%r⎪%p⎪%t⎪<swishdate fmt=/%A, %d. %B %Y/>⎪%d\n"
-x "<swishrank>\t<swishdocpath>\t<swishtitle>\t<keywords>\n
-x "xml_out: \<title\><swishtitle>\>\</title\>\n"
-x "xml_out: <swishtitle fmt='<title>%s</title>'>\n"

🌈 -H [0⎪1⎪2⎪3⎪<n>] (header output verbosity)

The -H n switch gives you extended header output 🎀. It's most useful when you're searching more than one index file at a time, by specifying more than one index file with the -f switch. -H 2 will generate a set of headers specific to each index file, so you get access to the settings used to generate each one 🔍.

Even when you're searching a single index file, -H n will give you extra info about that index file, how it was indexed, and how swish is interpreting your query 🧠.

• -H 0 — print no header information, output only search result entries.
• -H 1 — print standard result header (default).
• -H 2 — print additional header information for each searched index file.
• -H 3 — enhanced header output (e.g. print stopwords).
• -H 9 — print diagnostic information in the header of the results (changed from: C<-v 4>)

🔥 -R [0⎪1] (Ranking Scheme)

🧪 This is an experimental feature!

The default ranking scheme in SWISH-E checks each word in a query by its frequency and position in each document 📊. The default scheme is 0.

New in version 2.4.3, you can optionally pick an experimental ranking scheme that, on top of document frequency and position, uses Inverse Document Frequency (IDF), or the relative frequency of each word across all the indexes being searched. It also uses Relative Density, which is the normalization of a word's frequency in relationship to the number of words in the document 🧮.

📝 NOTE: IgnoreTotalWordCountWhenRanking must be set to no or 0 in your index(es) for -R 1 to work.

Specify -R 1 to turn on IDF ranking 🦋. See the API documentation for how to set the ranking scheme in your Perl or C program 💻.

---

✨ OTHER SWITCHES ✨

🎀 -V (version)

Print the current version 🏷️.

🔤 -k letter (print out keywords)

The -k switch is used for testing. It makes swish print out all keywords in the index that begin with that letter. You can enter -k '*' to generate a list of all words indexed by swish 📜.

💀 -D index file (debug index)

The -D option is no longer supported in version 2.2 ❌.

🐛 -T options (trace/debug swish)

The -T option prints out information that may be helpful when debugging swish-e's operation 🔍. This option replaced the "-D" option of previous versions 🔁.

Running "-T help" will print out a list of available options 📋.

---

🫶 Merging Index Files

In previous versions of Swish-e, indexing needed a very large amount of memory and could be super slow 🐌. Merging was the way around that: you indexed in chunks and then combined the indexes into a single index 🧩.

Indexing is much faster now and uses much less memory. With the "-e" switch, very little memory is needed to index a large site 🚀.

Still, it can be useful to merge different index files into one file for searching 💜. Maybe you want to keep separate site indexes plus a common one for a global search, or you have separate collections of documents you want to search all at once but manage separately 🗂️.

📦 -M index1 index2 ... indexN out_index

Merges the indexes specified on the command line. The last file name you enter is the output file. The output index must not exist, otherwise merge will not proceed 🛑.

Only indexes that were indexed with common settings may be merged 🤝. (e.g. don't mix stemming and non-stemming indexes, or indexes with different WordCharacter settings, etc.).

Use the "-e" switch while merging to reduce memory usage 🧠.

Merge generates progress messages regardless of the setting of "-v" 📣.

🗃️ -c configuration file

Specify a configuration file while indexing to add administrative information to the output index file 💅.

✨ Document Info ✨

okay so this part is basically the page's ID card 🪪 bestie. it's the revision stamp from the source file, a little fingerprint that tells you exactly which version of the doc you're looking at 🔍

$Id: SWISH-RUN.pod 1741 2005-05-17 02:22:40Z karman $

and then there's just a lonely . sitting at the end 🙈 that's literally it, no cap. nothing else to see here.

---

And that's a wrap, bestie! 🎀 This is SWISH-RUN(1), version 2.4.7, dated 2009-04-04. go forth and index things! 💖🦋