SWISH-CONFIG(1)
🎀 NAME 🎀
Okay bestie, welcome to SWISH-CONFIG(1) 💖, the Swish-E Documentation page that spills all the tea on configuration file directives ✨
SWISH-CONFIG — Configuration File Directives 💅
🌸 OVERVIEW 🌸
This doc is the full menu of every configuration directive you can use in Swish-e. Literally everything, no cap 🫶
🦋 CONFIGURATION FILE 🦋
So the config file is where you get main character control 👑 It decides what files Swish-e indexes, how they get indexed, and where the index gets written 📝
The config file is a plain text file made of comments, blank lines, and config directives. The order of the directives doesn't matter, so the vibes can be all over the place 🌈 Some directives can be used more than once. Others are one-and-done, and if you add another one, it just overwrites the one before it, so be careful 💀 Directive case doesn't matter either, so upper, lower, or mixed case are all giving ✨
Comments are any line that starts with a "#":
# This is a commentAs of 2.4.3 you can continue a line by putting a backslash as the very last character on the line 🔗:
IgnoreWords \
am \
the \
fooDirectives can take more than one parameter. If a single parameter has whitespace in it, wrap it in quotes (single or double) 💖 Inside quotes, a backslash escapes the next character:
ReplaceRules append "foo bar" <- define "foo bar" as a single parameterIf you need a quote character inside the value, either backslash-escape it or wrap the whole thing in quotes of the other type 🙈
Backslashes also have a special job in regular expressions 🥹:
FileFilterMatch pdftotext "'%p' -" /\.pdf$/That makes the dot a real dot instead of "match any character" 🦖 If you put the regex in quotes, you have to use double-backslashes:
FileFilterMatch pdftotext "'%p' -" "/\\.pdf$/"Swish-e turns the double backslash into a single backslash before passing the parameter to the regular expression compiler 🧚
Commented example config files live in the conf directory of the Swish-e distribution 📂
Some command line arguments can override directives in the config file. For how to run Swish-e, check out SWISH-RUN 🏃♀️, and for how to search your index (with examples!), check the SWISH-SEARCH page 🔍
You hand the config file to Swish-e with the "-c" switch, like so:
swish-e -c myconfig.confYou can also split your directives across different config files 🍰 That way you can have one master config for lots of indexes and smaller configs for each separate index. Specify the different config files on the command line with "-c" (see SWISH-RUN), or pull in another config file with the IncludeConfigFile directive below 💅
Usually the directives are grouped in some logical order. Directives that control the source of the documents go first, then the directives that control how each document is filtered or how its words get indexed. (The directives below are grouped in this order 📋)
- "Administrative Headers Directives" — add administrative info to the header of the index file 📝
- "Document Source Directives" — pick the source documents and the location of the index file 📂
- "Document Contents Directives" — control how a document's content gets indexed 🧾
- "Directives for the File Access method only" — only applicable to the File Access indexing method 📁
- "Directives for the HTTP Access Method Only" — likewise, these only apply to the HTTP Access method 🌐
- "Directives for the prog Access Method Only" — these only apply to the prog Access method ⚙️
- "Document Filter Directives" — a special section about using document filters with Swish-e 🧹
💖 Alphabetical Listing of Directives 💖
Here's the full alphabetical lineup. Every single one, no skipping 💅
AbsoluteLinks [yes⎪NO]BeginCharacters *string of characters*BumpPositionCounterCharacters *string*Buzzwords [*list of buzzwords*⎪File: path]CompressPositions [yes⎪NO]ConvertHTMLEntities [YES⎪no]DefaultContents [TXT⎪HTML⎪XML⎪TXT2⎪HTML2⎪XML2⎪TXT*⎪HTML*⎪XML*]Delay *seconds*DontBumpPositionOnEndTags *list of names*DontBumpPositionOnStartTags *list of names*EnableAltSearchSyntax [yes⎪NO]EndCharacters *string of characters*EquivalentServer *server alias*ExtractPath *metaname* [replace⎪remove⎪prepend⎪append⎪regex]FileFilter *suffix* *program* [options]FileFilterMatch *program* *options* *regex* [*regex* ...]FileInfoCompression [yes⎪NO]FileMatch [contains⎪is⎪regex] *regular expression*FileRules [contains⎪is⎪regex] *regular expression*FuzzyIndexingMode [NONE⎪Stemming⎪Soundex⎪Metaphone⎪DoubleMetaphone]FollowSymLinks [yes⎪NO]HTMLLinksMetaName *metaname*IgnoreFirstChar *string of characters*IgnoreLastChar *string of characters*IgnoreLimit *integer integer*IgnoreMetaTags *list of names*IgnoreNumberChars *list of characters*IgnoreTotalWordCountWhenRanking [YES⎪no]IgnoreWords [*list of stop words*⎪File: path]ImageLinksMetaName *metaname*IncludeConfigFileIndexAdmin *text*IndexAltTagMetaName *tagname*⎪as-textIndexComments [yes⎪NO]IndexContents [TXT⎪HTML⎪XML⎪TXT2⎪HTML2⎪XML2⎪TXT*⎪HTML*⎪XML*] *file extensions*IndexDescription *text*IndexDir [URL⎪directories or files]IndexFile *path*IndexName *text*IndexOnly *list of file suffixes*IndexPointer *text*IndexReport [0⎪1⎪2⎪3]MaxDepth *integer*MaxWordLimit *integer*MetaNameAlias *meta name* *list of aliases*MetaNames *list of names*MinWordLimit *integer*NoContents *list of file suffixes*obeyRobotsNoIndex [yes⎪NO]ParserWarnLevel [0⎪1⎪2⎪3]PreSortedIndex *list of property names*PropCompressionLevel [0-9]PropertyNameAlias *property name* *list of aliases*PropertyNames *list of meta names*PropertyNamesCompareCase *list of meta names*PropertyNamesIgnoreCase *list of meta names*PropertyNamesNoStripChars *list of meta names*PropertyNamesDate *list of meta names*PropertyNamesNumeric *list of meta names*PropertyNamesMaxLength integer *list of meta names*PropertyNamesSortKeyLength integer *list of meta names*ReplaceRules [replace⎪remove⎪prepend⎪append⎪regex]ResultExtFormatName name -x format stringSpiderDirectory *path*StoreDescription [XML <tag>⎪HTML <meta>⎪TXT size]SwishProgParameters *list of parameters*SwishSearchDefaultRule [<AND-WORD>⎪<or-word>]TmpDir *path*TranslateCharacters [*string1 string2*⎪:ascii7:]TruncateDocSize *number of characters*UndefinedMetaTags [error⎪ignore⎪INDEX⎪auto]UndefinedXMLAttributes [DISABLE⎪error⎪ignore⎪index⎪auto]UseStemming [yes⎪NO]UseSoundex [yes⎪NO]UseWords [*list of words*⎪File: path]WordCharacters *string of characters*XMLClassAttributes *list of XML attribute names*
✨ Directives that Control Swish ✨
These config directives control the general behavior of Swish-e 🎛️
🦋 IncludeConfigFile *path to config file*
This directive lets you pull in config directives that live in another file 🫶
IncludeConfigFile /usr/local/swish/conf/site_config.config🌈 IndexReport [0⎪1⎪2⎪3]
This is how chatty you want reporting to be while indexing 📢 You can pick any number from 0 to 3. 0 is totally silent and 3 is the most verbose 🗣️ The default is 1.
The "-v" switch on the command line can override this (see SWISH-RUN) 💅
🧚 ParserWarnLevel [0⎪1⎪2⎪3]
This sets the error level when you're using the libxml2 parser for XML and HTML. libxml2 will point out structural errors in your documents 🔍 The levels are:
0 = no report
1 = fatal errors
2 = errors
3 = warningsCurrently (as of 2.4.4 - early 2005) libxml2 only reports errors at level 2. The default as of 2.4.4 is "2", which should report any errors that might mean a document had trouble parsing 🚨
The exception is UTF-8 to Latin-1 conversion errors, which are reported at level 3 (changed from 1 in 2.4.4). Even though these can mean there's a problem indexing text, they only show up at level 3 because they can be super common 🙄
It's recommended that you index at ParserWarnLevel 3 when you're first starting out, so you can see what errors and warnings get reported 👀 Then lower the level once you understand which documents are causing parsing problems and why 💖
🌸 IndexFile *path*
IndexFile specifies where the generated index file lives 📁 If you don't set it, Swish-e will create a file called index.swish-e in the current directory.
IndexFile /usr/local/swish/site.index🍓 obeyRobotsNoIndex [yes⎪NO]
When this is enabled, Swish-e will skip indexing any HTML file that contains:
<meta name="robots" content="noindex">The default is to ignore these meta tags and index the document anyway. This tag is described at http://www.robotstxt.org/wc/exclusion.html 🤓
Note: This feature is only available with the libxml2 HTML parser 🚫
If you're using the libxml2 parser (HTML2 and XML2), you can also use these comments in your documents to prevent indexing:
<!-- SwishCommand noindex -->
<!-- SwishCommand index -->and/or these may be used too:
<!-- noindex -->
<!-- index -->These are super helpful for keeping common headers, footers, and menus out of the index 🙌
NOTE: The following items are currently not available. They require Swish-e to parse the configuration file while searching.
🚧 EnableAltSearchSyntax [yes⎪NO]
NOTE: This item is currently not available.
This enables alternate search syntax, which allows a basic "Altavista(c)", "Lycos(c)", etc.-style search. That means a search query can use "+" and "-" as syntax parameters ✨
Okay, so here's how the search operators look in action 💅:
swish-e -w "+word1 +word2 -word3 word4 word5""+"— the word right after it has to show up in every single doc that comes back 👑"-"— the word right after it can't be in ANY of the found docs 🙅" "— the word right after it just gets searched for, no strings attached 🦋
SwishSearhOperators <and-word> <or-word> <not-word>
💀 Heads up, bestie: this one is currently not available.
This config directive lets you swap out Swish-e's boolean search operators, which is so useful if you want them in your own language 🌍. The default is AND OR NOT.
Example (german) 🇩🇪:
SwishSearchOperators UND ODER NICHTSwishSearchDefaultRule [<AND-WORD>⎪<or-word>]
💀 Heads up, bestie: this one is currently not available.
SwishSearchDefaultRule picks the default Boolean operator to use when you don't specify one between words or phrases. The default is "AND" ✨.
The word you pick has to match one of the available SwishSearchOperators.
Example:
SwishSearchOperators UND ODER NICHT
# Make it act like a web search engine
SwishSearchDefaultRule ODERResultExtFormatName name -x format string
💀 Heads up, bestie: this one is currently not available.
Swish-e's output can be shaped with a format string, passed in using the -x command line argument. With ResultExtFormatName you can give a predefined format string a name 🏷️.
Examples:
ResultExtFormatName moreinfo "%c⎪%r⎪%t⎪%p⎪<author>⎪<publishyear>\n"Then when you're searching, you can call that format string by its name:
swish-e ... -x moreinfo ...See the -x switch in SWISH-RUN for more info about output formats 🌸.
🎀 Administrative Headers Directives
Swish-e stores config info in the header of the index file 📇. You can grab it back while searching, or with the functions in the Swish-e C library. There are a bunch of fields free for your own use. None of these fields are required, so no pressure 💖:
IndexNametextIndexDescriptiontextIndexPointertextIndexAdmintext
These are the vibes-and-info fields that help users and admins out 🫶. IndexName should be the name of your index, like a book title. IndexDescription is a short description of the index, or a URL pointing to a fuller one. IndexPointer should point to the original information, most likely a URL. IndexAdmin should be the name of the index maintainer and can include name and email info. Keep these values to about 70 characters or so, and wrap them in quotes ✨.
Also, FYI, the automatically generated date in index files is in D/M/Y and 24-hour format 🕰️.
Examples:
IndexName "Linux Documentation"
IndexDescription "This is an index of /usr/doc on our Linux machine."
IndexPointer http://localhost/swish/linux/index.html
IndexAdmin webmaster🎀 Document Source Directives
These directives control what documents get indexed and how they're accessed 📂. See also Directives for the File Access method only and Directives for the HTTP Access Method Only for the ones specific to those access methods 🦋.
IndexDir [directories or files⎪URL⎪external program]
IndexDir defines where Swish-e gets its documents from. Swish-e currently supports three file access methods: File system, HTTP (also called spidering), and prog, for reading files from an external program 🧚.
The -S command line argument selects the file access method:
swish-e -c swish.config -S fs - file system
swish-e -c swish.config -S http - internal http spider
swish-e -c swish.config -S prog - external program of any typeFor the fs method, IndexDir is a space-separated list of files and directories to index. On MS Windows, use a forward slash as the path separator 🪟.
For the http method, IndexDir is a list of space-separated URLs.
For the prog method, IndexDir is a list of space-separated programs to run (which generate documents for swish to index).
You can have more than one IndexDir directive 🙌.
Any sub-directories of any listed directory get indexed too 📁.
🙈 Note: While processing directories, Swish-e ignores any files or directories that begin with a dot ("."). If you want to index those, specify their name withIndexDiror-i.
Examples:
# Index this directory an any subdirectories
IndexDir /usr/local/home/http
# Index the docs directory in current directory
IndexDir ./docs
# Index these files in the current directory
IndexDir ./index.html ./page1.html ./page2.html
# and index this directory, too
IndexDir ../public_htmlFor the HTTP method of access, specify the URLs where you want the spidering to begin 🕸️.
Example:
IndexDir http://www.my-site.com/index.html
IndexDir http://localhost/index.htmlObviously, using the HTTP method to index is way slower than indexing local files 🐢. Be well aware that some sites don't appreciate spidering and may block your IP address 💀. You might want to contact the remote site before spidering their web site. More info on spidering is in Directives for the HTTP Access Method Only below.
For the prog method of access, IndexDir specifies the path to the program(s) to execute. The external program has to correctly format the documents it passes back to Swish-e. Examples of external programs are provided in the prog-bin directory 🗂️.
IndexDir ./myprogram.plSee prog for details.
⚠️ Note: Not all directives work with all methods.
NoContents *list of file suffixes*
Files with these suffixes won't have their contents indexed, but their path name (file name) will be indexed instead 🙅♀️📄.
If the file's type is HTML or HTML2 (as set by "IndexContents" or "DefaultContents"), then the file gets parsed for an HTML title, and that title is indexed. Note that you must set the file's type with "IndexContents" or "DefaultContents": ".html" and ".htm" are NOT type HTML by default 🚨. For example:
IndexContents HTML* .htm .htmlIf a title is found, it still gets checked for "FileRules title", and the file gets skipped if a match is found. See "FileRules".
If the file's type is not HTML, or it is HTML and no title is found, then the file's path gets indexed.
For example, this will let you search by image file name 🖼️:
NoContents .gif .xbm .au .mov .mpg .pdf .ps🚨 Note: Using this directive will NOT cause files with those suffixes to be indexed. So if you use "IndexOnly" to limit the types of files that get indexed, you must specify the same suffixes in "IndexOnly" that are listed in "NoContents".
This does not work:
# Wrong!
IndexOnly .htm .html
NoContents .gif .xbm .au .mov .mpg .pdf .psA "-S prog" program can set the "No-Contents:" header to turn this feature on for a specific document (although it would be smarter for the "-S prog" program to simply only send the pathname or title to be indexed 🧠.
ReplaceRules [replace⎪remove⎪prepend⎪append⎪regex]
ReplaceRules lets you make changes to file pathnames before they're indexed 🔁. Those changed file names or URLs are what show up in search results.
For example, you might index your files locally (with the File system indexing method) but still return a URL in search results. This directive can be used to map the file names to their respective URLs on your web server 🌐.
There are five operations you can specify: replace, append, remove, prepend, and regex. They parse the pathname in the order you've typed them 🔢.
This directive uses C library regex.h regular expressions 🤓.
replace "the string you want replaced" "what to change it to"
remove "a string to remove"
prepend "a string to add before the result"
append "a string to add after the result"
regex "/search string/replace string/options"Remember, quotes are needed if an expression contains white space, and backslashes have special meaning 🥹.
Regex is an Extended Regular Expression. The first character found is the delimiter (but it's not smart enough to use matched chars such as [], (), and {}) 🙃.
The replace string may use substitution variables:
$0 the entire matched (sub)string
$1-$9 returns patterns captured in "(" ")" pairs
$` the string before the matched pattern
$' the string after the matched patternThe options change the behavior of the expression:
i— ignore the case when matching 🔤g— repeat the substitution for the entire pattern 🔁
Examples:
ReplaceRules replace testdir/ anotherdir/
ReplaceRules replace [a-z_0-9]*_m.*\.html index.html
ReplaceRules remove testdir/
ReplaceRules prepend http://localhost/
ReplaceRules append .html
ReplaceRules regex !^/web/(.+)/!http://$1.domain.com/!
replaces a file path:
/web/search/foo/index.html
with
http://search.domain.com/foo/index.html
ReplaceRules regex #^#http://localhost/www#
ReplaceRules prepend http://localhost/www (same thing)
# Remove all extensions from C source files
ReplaceRules remove .c # ERROR! That "." is *any char*
ReplaceRules remove \.c # much better...
ReplaceRules remove "\\.c" # if in quotes you need double-backslash!
ReplaceRules remove "\.c" # ERROR! "\." -> "." and is *any char*IndexContents [TXT⎪HTML⎪XML⎪TXT2⎪HTML2⎪XML2⎪TXT*⎪HTML*⎪XML*] file extensions
The IndexContents directive assigns one of Swish-e's document parsers to a document, based on its extension 🧩. Swish-e currently knows how to parse TXT, HTML, and XML documents.
The XML2, HTML2, and TXT2 parsers are currently only available when Swish-e is configured to use libxml2 💜.
You can use XML*, HTML*, and TXT* to select the parser automatically. If libxml2 is installed, then it gets used to parse the content. Otherwise, Swish-e's internal parsers get used 🌈.
If a document doesn't get a parser from IndexContents, bestie, it's using the HTML2 parser by default 💅 as long as libxml2 is installed. Otherwise it falls back to Swish-e's internal HTML parser 🌸. The DefaultContents directive lets you pick a parser for docs that don't match any extension you set with IndexContents ✨.
IndexContents HTML* .htm .html .shtml
IndexContents TXT* .txt .log .text
IndexContents XML* .xmlHTML* is the default type for every file unless you say otherwise (and DefaultContents can change that default) 🎀. Swish-e pulls titles out of HTML files when they're available 📝 and tracks where text sits in the document so context searching works 🦋. Check out -t in SWISH-RUN for that part.
If you use filters (through the FileFilter directive) to convert documents, you also need to add those extensions, bestie 💖. Say you're converting .pdf to .html. You have to tell Swish-e that .pdf should go through the internal HTML parser:
FileFilter .pdf pdf2html
IndexContent HTML .pdfSee also Document Filter Directives 🌈.
Note: some of this might switch to content-types instead of file extensions in the future 👀. Look at SWISH-3.0 for that.
DefaultContents [TXT⎪HTML⎪XML⎪TXT2⎪HTML2⎪XML2⎪TXT*⎪HTML*⎪XML*]
This sets the default parser for any document not listed in IndexContents 🍓. If you don't set it, the default is HTML.
The XML2, HTML2, and TXT2 parsers are only around when Swish-e is built to use libxml2 🧚.
You can use XML*, HTML*, and TXT* to let the parser get picked automatically 🤖. If libxml2 is installed, that one parses the content. If not, Swish-e's internal parsers take over.
Example:
DefaultContents HTMLUse DefaultContents when you're spidering, okay? HTML files can come back with no file extension at all, like when you request a directory and get the default index.html back 🏠.
FileInfoCompression [yes⎪NO]
💀 This directive is currently not supported. 💀
Setting FileInfoCompression to "yes" squishes the index file to save disk space 🗜️. That might make indexing take longer. The default is "no".
Also check out the -e switch in SWISH-RUN if you want to save RAM during indexing 🧠.
✨ Document Contents Directives ✨
These directives decide what info gets pulled out of your source documents, and how that info shows up when you search 👑.
ConvertHTMLEntities [YES⎪no]
Swish-e can automatically convert ASCII entities while indexing HTML documents (this doesn't apply to HTML2) 💅. For speed, you can set this to "no" if your docs don't have any HTML entities. The default is "yes".
If ConvertHTMLEntities is "no", the entities get indexed exactly as they are, no conversion 🙈.
NOTE: Entities inside XML files and files parsed with libxml2 (HTML2) get converted no matter what this setting says 💅.
MetaNames *list of names*
META names let you define "fields" in your XML and HTML documents 🎀. Then you can use them in queries to only search the words inside that META name. Like, say you've got a META tagged field called "subjects" 🦋. You can search for "foo" and only get back docs where "foo" shows up in the "subjects" META tag:
swish-e -w subjects=foo(Also check out the -t switch in SWISH-RUN for context searching in HTML documents 🌸.)
The MetaNames directive takes a space separated list 📝. For example:
MetaNames meta1 meta2 keywords subjectsYou can also use UndefinedMetaTags to automatically pull meta names out of your HTML and XML documents, and to skip indexing the content of meta tags 🙅♀️.
META tags can look two ways in your HTML source docs 👀:
<META NAME="meta1" CONTENT="some content">and (if you're using the HTML2/libxml2 parser):
<meta1>
some content
</meta1>But that second version is invalid HTML 😭, so you'll get a warning if ParserWarningLevel is set (libxml2 only).
In XML documents, use this format:
<meta1>
Some Content
</meta1>Then you can limit your search to just META meta1 like this 🔥:
swish-e -w 'meta1=(apples or oranges)'You can nest the XML and the start/end tag versions, too 🧚:
<keywords>
<tag1>
some content
</tag1>
<tag2>
some other content
</tag2>
<keywords>Then you can search in both tag2 and tag2 with:
swish-e -w 'keywords=(query words)'Swish-e indexes all text as some metaname 💜. The default is "swishdefault", so these two queries are the same:
swish-e -w foo
swish-e -w swishdefault=fooWhen it indexes HTML, Swish-e indexes the HTML title as default text ✨. So when you search, you'll find matches in both the HTML body and the HTML title. Swish also indexes the content of meta tags by default 🌈. So:
swish-e -w foowill find "foo" in the body, the title, or any meta tags 🫶.
Right now there's no way to stop Swish-e from indexing the title contents along with the body contents 🙃. But check out "UndefinedMetaTags" if you want to control how meta tags get indexed.
If you only want to search the title text, use this 👑:
MetaNames swishtitleThis indexes the title text separately under the built-in swish internal meta name "swishtitle" 🎀. Then you can search like this:
swish-e -w foo -- search for "foo" in title, body (and undefined meta tags)
swish-e -w swishtitle=foo -- search for "foo" in title onlyOn top of swishtitle, you can limit searches to a document's path 🛤️:
MetaNames swishdocpathThen to search for "foo" and also limit results to documents that have "manual" or "tutorial" in their path 🦋:
swish-e -w foo swishdocpath=(manual or tutorial)See also "ExtractPath" 🌸.
MetaNameAlias *meta name* *list of aliases*
MetaNameAlias gives a meta name some aliases 💅. Say your docs have meta tags "description", "summary", and "overview" that all summarize the doc. You could do this:
MetaNames summary
MetaNameAlias summary description overviewThen all three tags get indexed as the meta tag "summary" 🍓. You can search all of those fields like this:
-w summary=fooAliases work at search time too, so these also limit the search to the "summary" meta name 🔥:
-w description=foo
-w overview=fooMetaNamesRank integer *list of meta names*
You can give meta names a bias that changes how ranking gets calculated 📈. The range is -10 to +10, and zero means no bias ⭐.
MetaNamesRank 4 subject
MetaNamesRank 3 swishdefault
MetaNamesRank 2 author publisher
MetaNamesRank -5 wrongwords🧪 This feature is still considered experimental. If you use it, please send feedback to the discussion list 💜.
HTMLLinksMetaName *metaname*
This lets you index HTML links 🔗. By default, HTML links (href tags) are not indexed by Swish-e 🙅♀️. This directive defines a metaname, and links get indexed under that meta name.
Example:
HTMLLinksMetaName linksNow, if you want to limit searches to files that link to "home.html", do this 🏡:
-w links='"home.html"'The double quotes force a phrase search 💅.
If you want Swish-e to index links as normal text, use:
HTMLLinksMetaName swishdefaultThis feature is only available with the libxml2 HTML parser 🧚.
ImageLinksMetaName *metaname*
This lets you index image links under a metaname 🖼️. By default, image URLs are not indexed.
Example:
ImagesLinksMetaName imagesNow, if you want to find pages with a nice image of a beach 🏖️:
-w images='beach'If you want Swish-e to index links as normal text, use:
ImageLinksMetaName swishdefaultThis feature is only available with the libxml2 HTML parser 🌊.
IndexAltTagMetaName *tagname*⎪as-text
This lets you index the ALT text of <IMG> tags 🖼️. You can give it a tag name, which gets used as a metaname, or use the special text "as-text" to index the ALT text as if it were plain text at the current location 💖.
For example, with a tag name:
IndexAltTagMetaName barmakes this markup:
<foo>
<img src="/someimage.png" alt="Alt text here">
</foo>look like this:
<foo>
<bar>Alt text here</bar>
</foo>Then the normal rules ("MetaNames" and "PropertyNames") apply to how that text gets indexed ✨.
If you use the special tag "as-text", then this:
<foo>
<img src="/someimage.png" alt="Alt text here">
</foo>simply becomes this:
<foo>
Alt text here
</foo>This feature is only available with the libxml2 parser (HTML2 and XML2) 🧚.
AbsoluteLinks [yes⎪NO]
If this is set to yes, Swish-e will try to turn relative URIs pulled from HTML documents into absolute URIs 🌐. That's for use with "HTMLLinksMetaName" and "ImageLinksMetaName". Swish-e uses any <BASE> tag it finds in the document. If there isn't one, it uses the file's pathname. The pathname used is the one after "ReplaceRules" has been applied to the document's pathname 🗺️.
Say you want to index image links under the metaname "images":
ImageLinksMetaName imagesIf an image lives in http://localhost/vacations/france/index.html and "AbsoluteLinks" is set to no, then an image in that document like this:
<img src="beach.jpeg">will only index "beach.jpeg" 😭.
But if you want more detail when searching, enable "AbsoluteLinks". Then Swish-e will index "http://localhost/vacations/france/beach.jpeg" 🥹. Now you can look for pictures of beaches, but only in France 🇫🇷:
-w images=(beach and france)This also means you can search for any images within France, bestie 🗼
-w images=(france)⚠️ Heads up: this feature is only available with the libxml2 HTML parser. 🙈
UndefinedMetaTags 🎀
UndefinedMetaTags [error⎪ignore⎪INDEX⎪auto]Okay so UndefinedMetaTags decides what Swish-e does during indexing when it finds a meta name that isn't listed in MetaNames 👀 There are four choices, and each one has its own energy:
error— if a meta name shows up that isn't inMetaNames, indexing gets halted and an error is reported 🚫ignore— the contents of that meta tag get ignored and not indexed, unless a metaname has been defined with theMetaNamesdirectiveindex— the contents of the meta tag get indexed, but they land in the main index unless there's already an enclosing metatag in force. This one is the default 💅auto— this one creates meta tags automatically for HTML meta names and XML elements. Using it is the same as listing every meta name explicitly in aMetaNamesdirective ✨
UndefinedXMLAttributes 🦋
UndefinedXMLAttributes [DISABLE⎪error⎪ignore⎪index⎪auto]This is basically the cousin of UndefinedMetaTags, but it only applies to XML documents (the ones parsed with libxml2). It lets you index attribute content, and gives you a way to index that content under a metaname. Like, UndefinedXMLAttributes can turn this:
<person age="23">
John Doe
</person>into this, as far as swish is concerned:
<person>
<person.age>
23
</person.age>
John Doe
</person>What happens to the 23 depends on how UndefinedXMLAttributes is set:
disable— XML attributes aren't parsed and aren't indexed. This is the default 🙅♀️error— if the concatenated meta name (e.g.person.age) isn't listed inMetaNames, indexing gets halted and an error is reported 🚫ignore— the contents of the meta tag are ignored and not indexed, unless a metaname has been defined with theMetaNamesdirectiveindex— the contents of the meta tag are indexed, but they go in the main index unless there's an enclosing metatag already in forceauto— this creates meta tags from the combined element and attributes (plus the XML class name). Use it with caution, because it can generate a LOT of metaname entries 😭
Also peep the example below under XMLClassAttribues 👀
XMLClassAttributes
XMLClassAttributes *list of XML attribute names*Okay this one is so cute 🧚 It combines an XML class name with the element name to make up a metaname. For example:
XMLClassAttributes class
<person class="first">
John
</person>
<person class="last">
Doe
</person>Swish-e sees that as:
<person>
<person.first>
John
</person.first>
</person>
<person>
<person.last>
Doe
</person.last>
</person>How the data gets indexed depends on MetaNames and UndefinedMetaTags 💜
Here's an example using this config, which combines XMLClassAttributes and UndefinedXMLAttributes:
XMLClassAttributes class
UndefinedMetaTags auto
UndefinedXMLAttributes auto
IndexContents XML2 .xmlThe source XML file looks like:
<xml> <person class="student" phone="555-1212" age="102"> John </person>
<person greeting="howdy">Bill</person> </xml>Swish-e parses as:
./swish-e -c 2 -i 1.xml -T parsed_tags parsed_text -v 0
Indexing Data Source: "File-System"
<xml> (MetaName)
<person> (MetaName)
<person.student> (MetaName)
<person.student.phone> (MetaName)
555-1212
</person.student.phone>
<person.student.age> (MetaName)
102
</person.student.age>
John
</person>
<person> (MetaName)
<person.greeting> (MetaName)
howdy
</person.greeting>
Bill
</person>
</xml>
Indexing done!One thing to note 🫶 The first <person> block finds a class name "student", so all the metanames created from attributes use the combined name person.student. The second <person> block doesn't contain a "class", so the attribute name gets combined directly with the element name (e.g. person.greeting).
ExtractPath 🗂️
ExtractPath *metaname* [replace⎪remove⎪prepend⎪append⎪regex]This directive lets you index extracted parts of a document's path. A common use is to limit searches to specific areas of your file tree, which is so useful, honestly 🔥
The extracted string gets indexed under the metaname you specify.
See ReplaceRules for the rundown on the various pattern replacement methods, but you'll be using the regex method.
For example, say your file system (or web tree) is organized into departments:
/web/sales/foo...
/web/parts/foo...
/web/accounting/foo...And you want a way to limit searches to just the documents under "sales":
ExtractPath department regex !^/web/([^/]+)/.*$!$1!Which says: extract the department name (as substring $1) and index it as the metaname "department". Then, to limit a search to the sales department:
swish-e -w foo AND department=salesNote that the regex method uses a substitution pattern, so to index only a sub-string, match the entire document path in the regular expression, as shown above. Otherwise, any part that isn't matched will end up in the substitution pattern 🙈
See the ExtractPathDefault option for a way to set a value if no patterns match.
Although unlikely, you may use more than one ExtractPath directive. More than one directive with the same metaname will operate successively (in the order listed in the configuration file) on the path. This lets you run regular expressions on the results of the previous pattern substitution, kind of like piping the output from one expression into the pattern of the next:
ExtractPath foo regex !^(...).+$!$1!
ExtractPath foo regex !^.+(.)$!$1!So, the third letter is indexed as the metaname "foo" if both patterns match 🧠
ExtractPath foo regex !^X(...).+$!$1!
ExtractPath foo regex !^.+(.)$!$1!Now (no X, no match!), if the first pattern doesn't match, the last character of the path name is indexed instead. You must be clear on this behavior if you're using more than one ExtractPath directive with the same metaname 💀
The document path operated on is the real path swish used to access the document. That means the ReplaceRules directive has no effect on the path used with ExtractPath.
The full path is used for each metaname if more than one ExtractPath directive is used. That means changes to the path used in ExtractPath foo don't affect the path used by ExtractPath bar ✨
ExtractPathDefault 🌸
ExtractPathDefault *metaname* default_valueThis can be used with ExtractPath to set a default string to index under the given metaname, for when none of the ExtractPath patterns match.
For example, say you want to index each document with a metaname "department" based on these path examples:
/web/sales/foo...
/web/parts/foo...
/web/accounting/foo...But you're also indexing documents that don't follow that pattern, and you want to search those separately too:
ExtractPath department regex !^/web/([^/]+)/.*$!$1!
ExtractPathDefault department otherNow you can search like this:
-w foo department=(sales) - limit searches to the sales documents
-w foo department=(parts) - limit searches to the parts documents
-w foo department=(accounting) - limit searches to the accounting documents
-w foo department=(other) - everything but sales, parts, and accounting.This basically is a shortcut for:
-w foo not department=(sales or parts or accounting)...but you don't need to keep track of what got extracted 💖
PropertyNames 🌈
PropertyNames *list of meta names*
PropertyNamesCompareCase *list of meta names*
PropertyNamesIgnoreCase *list of meta names*Swish-e lets you specify certain META tags that can be used as document properties. The contents of any META tag that's been identified as a document property can be returned as part of the search results, along with the rank, file name, title, and document size (see the -p and -x switches in SWISH-RUN) 📄
Properties are great for returning extra data from documents in search results. This saves the effort of reading and parsing the source files while you're reading Swish-e search results, and it's especially useful when the source documents are no longer available or slow to access (e.g. over http) 🥹
Another feature of properties is that Swish-e can use PropertyNames for sorting search results (see the -s switch).
PropertyNames author subjectsThere are two variations: PropertyNamesCompareCase and PropertyNamesIgnoreCase. These tell Swish-e to either ignore or compare case when sorting results. The default for PropertyNames is to ignore case.
PropertyNamesIgnoreCase subject
PropertyNamesCompareCase keywordThe defaults for "internal" properties are:
swishtitle -- ignore the case
swishdocpath -- compare case
swishdescription -- compare caseThese can be overridden with PropertyNamesCompareCase and PropertyNamesIgnoreCase:
PropertyNamesCompareCase swishtitleHeads up, though: using PropertyNames will increase the size of your index files, sometimes significantly 📈 Properties will be compressed if Swish-e is compiled with zlib, as described in the INSTALL manual page.
If Swish-e finds more than one property of the same name in a document, the property's contents will be concatenated for strings, and a warning will be issued for numeric (or date) properties 🔥
💅 PropertyNamesNoStripChars 💖
okay bestie, PropertyNamesNoStripChars tells swish-e that the listed properties should not have their low ASCII characters swapped out for a space 🫶. They get stored exactly as they show up in the document, no edits 💅
When you print properties with the swish-e binary, newlines get turned into a space ✨ If you want them left alone, use the swish-e library (or the SWISH::API perl module) to grab the properties without newlines replaced 🦋
PropertyNamesNumeric 🔢
this one is basically PropertyNames but with a glow-up 💅 It flags the property as a string of digits (an integer value) and stores it as binary data instead of a string 🤯 That's what lets -s sort and -L limit the property correctly ✨
swish-e uses strtoul(3) to turn the string into an unsigned long integer, so only positive integers can be stored 🙈
future versions of swish-e might be able to store other property types, like negative integers and real numbers 👀 This directive may change in future releases of Swish-e, so keep an eye on it 🌈
PropertyNamesDate 📅
this is exactly like PropertyNamesNumeric, but it also marks the number as a machine timestamp (seconds since Epoch) 🕰️ and it prints a formatted date whenever it returns this property. see -x in SWISH-RUN for more 🎀
swish-e will not parse dates when indexing, so you have to hand it a timestamp 💀
PropertyNameAlias property name list of aliases 🧚
PropertyNameAlias lets a property name have aliases, which is so iconic 👑 For example, say you're indexing HTML files plus XML files written in English, German, and Spanish. Those use the tags "title", "titel", and "título", so you can do this:
PropertyNameAlias swishtitle title titel título tituloheads up: swishtitle is the built-in property that stores a document's title 🍓 So you don't need to list it as a PropertyName before using it 💖
PropertyNamesMaxLength integer list of meta names 📏
this one sets the max length of the text stored in a property ✨ You give it a number between 0 and the max integer size on your platform, plus a list of properties. The properties you name must not be aliases 🙅♀️
if any of the property names don't exist yet, swish-e creates them for you 🫶 (so you don't need to define them with PropertyNames first)
honestly this is mostly useful when parsing HTML or XML with the libxml2 parser 💜
for example:
PropertyNamesMaxLength 1000 swishdescription
PropertyNameAlias swishdescription bodyis kinda like this:
StoreDescription HTML <body> 1000
StoreDescription XML <body> 1000
StoreDescription HTML2 <body> 1000
StoreDescription XML2 <body> 1000but StoreDescription lets you set the tag for each parser type, so it's a bit more hands-on 🎀
and this one collects all the heading text into a single property called "headings", not going over 1000 characters 📚
PropertyNamesMaxLength 1000 headings
PropertyNameAlias headings h1 h2 h3 h4PropertyNamesSortKeyLength integer list of meta names ⏱️
this sets the length of the string used when sorting 🌸 The default is 100 characters. the -T metanames debugging option will list the current values for an index, so you can check your work 🔍
it's used when sorting during indexing, and maybe when sorting while searching too 🦋 It also affects the order when you limit to a range of values with the -L option ✨
PreSortedIndex list of property names 🗂️
by default, swish-e generates presorted tables while indexing for each property name 💅 That makes sorting faster when it's time to generate results 🏃♀️ But on large document collections, presorting can add to the indexing time and to the total size of the index 😭 This directive lets you decide exactly which properties get presorted 👑
- if
PreSortedIndexisn't in the config file (the default), all the properties get presorted at indexing time 🌈 - if it's present without any parameter, no properties get presorted 🙈
- otherwise, only the property names you list get presorted 💖
say you only want to sort results by a property called "title":
PropertyNames title age time
PreSortedIndex titleStoreDescription 📝
StoreDescription [XML <tag> size⎪HTML <meta> size⎪TXT size]this lets you store a document description in the index file 🫶 That description can show up in your search results when you use -x to include the swishdescription for extended results, or when you use -p swishdescription ✨
the document type (XML, HTML, or TXT) has to match the type currently being indexed, as set by IndexContents or DefaultContents 🚨 See those directives for the possible values. A super common problem is using StoreDescription without setting the document type with IndexContents or DefaultContents 💀 Another sneaky problem is mismatched types:
IndexContents HTML2 .html
StoreDescription HTML <body>here, .html documents get assigned a type of HTML2 (and get parsed by the libxml2 parser) 🧚 But the description won't be stored, because it's type HTML instead of HTML2 😭
for text documents, you specify the type TXT (or TXT2 or TXT*) and how many characters to capture 📖
StoreDescription TXT 20that stores only the first twenty characters from the text file in the swish-e index file 🌷
for HTML and XML file types, you specify the tag to use for the description, and optionally the number of characters 🎀 If you don't specify a number, it captures the entire contents of the tag 🍓
StoreDescription HTML <body> 20000
StoreDescription XML <desc> 40again, documents must be assigned a document type with IndexContents or DefaultContents to use this feature 💅
swish-e will compress the descriptions (or any other large property) if it's compiled to use zlib (see INSTALL) ✨ This is recommended when you're using StoreDescription with a large number of documents 🫧 Compression of 30% to 50% isn't uncommon with HTML files 🤩
PropCompressionLevel [0-9] 🗜️
this sets the compression level used when storing properties to disk 💜 A setting of zero means no compression, and a setting of nine means the most compression 🔥
the default depends on the default setting compiled with zlib, but it's typically six 🌈
this is super useful when you use StoreDescription to store a lot of text in properties, or when you use PropertyNames with large property sizes 📚
properties have to be over a value defined in config.h (100 is the default) before compression will even be attempted 🙈 And swish-e will never store the result of compression if the compressed data is larger than the original data 🫶
this option is only available when swish-e is compiled with zlib support, so check your build first ⚠️
TruncateDocSize number of characters ✂️
TruncateDocSize limits the size of a document while indexing documents and/or using filters 🦋 It truncates the number of bytes read from a document down to the size you specify. So if a document is bigger, swish-e only reads that many bytes 🌸
example:
TruncateDocSize 10000000the default is zero, which means read all the data 💖
Warning: if you use TruncateDocSize, use it with care! 💀 It's a safety belt only, meant to limit things like filter output, when accessing databases, or to stop "runaway" filters 🚨 Truncating doc input may destroy document structures for swish-e (e.g. swish may miss closing tags for XML or HTML documents) 😭
TruncateDocSize doesn't currently work with the "prog" input source method 🙅♀️
FuzzyIndexingMode NONE⎪Stemming⎪Soundex⎪Metaphone⎪DoubleMetaphone 🧠
FuzzyIndexingMode NONE⎪Stemming⎪Soundex⎪Metaphone⎪DoubleMetaphonethis one selects the type of index to create 🔮 Only one type of index can be created per index, so pick wisely 💅
it's a really good idea to create both a normal index and a fuzzy index, then let your search interface choose which one to use 🫶 Lots of people find fuzzy searches to be too fuzzy, lowkey 💀
if you want to see the available fuzzy indexing options, run:
swish-e -T LIST_FUZZY_MODESavailable options include:
None— words are stored in the index without any conversion. this is the default 🌸Stemming_*— uses one of the installed Snowball stemmers (http://snowball.tartarus.org/) 🦋 To see which stemmers are installed, runswish-e -T LIST_FUZZY_MODES(same command as above) ✨ For example, to use the Spanish stemming module:
FuzzyIndexingMode Stemming_esStemorStemming_en— this option is no longer supported. 💀 It selects the legacy swish-e English stemmer, and it's deprecated in favor of the Snowball English stemmerStemming_en1🥀
the legacy English stemmer converts words using the Porter stemming algorithm, from http://www.tartarus.org/~martin/PorterStemmer/ 📜 The Porter stemmer is a process for removing the commoner morphological and inflexional endings from English words. Its main use is as part of a term normalisation process, usually done when setting up Information Retrieval systems 🧚
so, for example, this helps a search for "running" also find "run" and "runs" 🏃♀️
but here's the tea ☕ the stemmer doesn't turn words into their root. It just programmatically chops off endings to make similar words with different endings stem to the same string of characters 🫧 It's not a perfect system, so searches on stemmed indexes often return curious results 🙈 For example, two entirely different words might stem to the same word 💀
stemming can also get confusing with wildcards (truncation) 😭 You might expect to find "running" by searching for runn*. But that fails on a stemmed index, because "running" stems to "run", while searching for runn* looks for words that start with "runn" 💔
✨ Soundex ✨
Soundex was cooked up in the 1880s so records for people with similar sounding names could be found more easily 💅. It's a coded surname based on how a surname sounds instead of how it's spelled 🦋. Names that sound alike, like Smith and Smyth, get filed together under the same Soundex code 👑. This is mostly useful for US English 🌸.
Soundex should not be used to search for sound-alike words, bestie 🙈. Metaphone is way more appropriate for generic sound matching of words. Soundex should only be used when you need to search multiple documents for proper names that sound similar. It's primarily used for indexing genealogical records 📜, and it may be useful for indexing other collections of data that are mostly names 🎀. Lots of common name variations get matched by Soundex. The only notable exception is the first letter of the name, which is not matched for sound 💀.
🧚 Metaphone and DoubleMetaphone 🧚
Words get transformed into a short series of letters representing the sound of the word (in English) ✨. Metaphone algorithms are often used for looking up mis-spelled words in dictionary programs, so they're super helpful for that 💖.
Source: "From: http://aspell.sourceforge.net/metaphone/"
Lawrence Philips' Metaphone Algorithm is basically an algorithm that returns the rough approximation of how an English word sounds 🌈.
The "DoubleMetaphone" mode will sometimes generate two different metaphones for the same word 👯. It's supposed to be useful when a word may be pronounced more than one way 🫶.
A metaphone index should give results somewhere in between Soundex and Stemming 💜.
UseStemming [yes⎪NO]
Set it to yes to turn on the word stemming algorithm while indexing, or no to skip it 🌿.
UseStemming no
UseStemming yesWhen UseStemming is set to "yes", every word gets stemmed before it goes into the index ✨.
⚠️ This option is deprecated 😭. It has been superseded by FuzzyIndexingMode.
UseSoundex [yes⎪NO]
When UseSoundex is set to "yes", every word gets converted to a Soundex code before it goes into the index 🔮.
⚠️ This option is deprecated 💀. It has been superseded by FuzzyIndexingMode.
IgnoreTotalWordCountWhenRanking [YES⎪no]
Put yes to ignore the total number of words in the file when ranking gets calculated 🍓. It's often better with merges and small files 🧁. Default is yes.
IgnoreTotalWordCountWhenRanking noThe default was changed from no to yes in version 2.2 🎀.
NOTE: must be set tonoif you intend to use the-R 1option when searching 🚨.
MinWordLimit integer
Sets the minimum length of a word. Shorter words won't be indexed 🦋. The default is 1 (as defined in src/config.h).
MinWordLimit 5MaxWordLimit integer
Sets the maximum length of an indexable word. Every longer word won't be indexed 🚫. The default is 40 (as defined in src/config.h).
Word-shaping crew 💅
WordCharactersstring of charactersIgnoreFirstCharstring of charactersIgnoreLastCharstring of charactersBeginCharactersstring of charactersEndCharactersstring of characters
These settings define what a word consists of to the Swish-e indexing engine 🧠. Compiled-in defaults live in src/config.h.
When indexing, Swish-e uses WordCharacters to split the document into words ✂️. A word is any string of non-blank characters that only contains characters listed in WordCharacters. If a string includes a character that isn't in WordCharacters, the word gets split into two or more separate words 💔.
For example:
WordCharacters abdeThis would turn "abcde" into two words, "ab" and "de" 🥹.
Next, any characters defined in IgnoreFirstChar get stripped off the start of each word, and IgnoreLastChar characters get stripped off the end 🧹. This lets you keep periods within a word (www.slashdot.com) but not at the end of one ✨. Characters in IgnoreFirstChar and IgnoreLastChar must be in WordCharacters.
Finally, the resulting words MUST begin with one of the characters listed in BeginCharacters and end with one of the characters listed in EndCharacters 🚦. BeginCharacters and EndCharacters must be a subset of the characters in WordCharacters. Often, WordCharacters, BeginCharacters and EndCharacters will all be the same 👯♀️.
The same process applies to the query while searching, so it's all consistent 🫶.
Getting these settings right takes careful thought and practice 😤. It's super helpful to create an index of a single test file and then look at the words that land in the index (see the "-v 4", "-D" and "-k" searching switches) 🔍.
Currently there's only support for eight-bit characters 🌈.
Example:
WordCharacters .abcdefghijklmnopqrstuvwxyz
BeginCharacters abcdefghijklmnopqrstuvwxyz
EndCharacters abcdefghijklmnopqrstuvwxyz
IgnoreFirstChar .
IgnoreLastChar .So the string:
Please visit http://www.example.com/path/to/file.html.will be indexed as the following words:
please
visit
http
www.example.com
path
to
file.htmlWhich means you can search for "www.example.com" as a single word, but searching for just "example" will not find the document 🙈.
Note: when indexing HTML documents, HTML entities get converted to their character equivalents before being processed with these directives 🎀. This is a change from previous versions of Swish-e, where you had to include the characters "0123456789&#;" to index entities. See also "ConvertHTMLEntities".
Buzzwords [list of buzzwords⎪File: path]
The Buzzwords option lets you name words that will be indexed no matter what WordCharacters, BeginCharacters, EndCharacters, stemming, soundex and many of the other word checks say 🚀. Main character energy!
Buzzwords are case insensitive 🫠.
Buzzwords should be separated by spaces and can span multiple directives 🧵. If the special format "File:filename" is used, the Buzzwords get read from an external file during indexing 📂.
Examples:
Buzzwords C++ TCP/IP
Buzzwords File: ./buzzwords.lstIf a Buzzword contains search operator characters, they must be backslashed when searching 🙃. For example:
Buzzwords C++ TCP/IP web=http
./swish-e -w 'web\=http'Buzzwords are found by splitting the text on whitespace, removing "IgnoreFirstChar" and "IgnoreLastChar" characters from the word, and then comparing with the list of "Buzzwords" 🔎. So if you're adding Buzzwords to an index, you'll probably want to define IgnoreFirstChar and IgnoreLastChar settings 💅.
Note: Buzzwords-specific settings for IgnoreFirstChar and IgnoreLastChar may be used in the future 🔮.
CompressPositions [yes⎪NO]
This option turns on zlib compression for individual word data in the index file 🗜️. The default is NO, which means the index word data is not compressed by default.
Enabling this can reduce the size of the index file, but it comes at the expense of slower wildcard search times 🐢.
The default changed from YES to NO starting with version 2.4.3 🍓.
IgnoreWords [list of stop words⎪File: path]
The IgnoreWords option lets you specify words to ignore, called stopwords 🚫. The default is to not use any stopwords 🌸.
Words should be separated by spaces and can span multiple directives 🧵. If the special format "File:filename" is used, the stop words get read from an external file during indexing 📂.
In previous versions of Swish-e you could use the directive:
IgnoreWords swishdefault - obsolete!to include a default list of compiled-in stopwords 🕰️. This keyword is no longer supported 💀.
Examples:
IgnoreWords www http a an the of and or
IgnoreWords File: ./stopwords.deUseWords [list of words⎪File: path]
UseWords defines the words that Swish-e will index 🎯. Only the words listed will be indexed ✨.
You can list words right after the directive (you can use more than one UseWords directive in a config file), and/or use the "File:" form to point at a file containing the words 📃:
UseWords perl python pascal fortran basic cobal php
UseWords File: /path/to/my/wordlistPlease drop the Swish-e list a note if you actually use this feature 💌. It may be removed from future versions 😭.
IgnoreLimit integer integer
This automatically omits words that show up too often in the files (these words are called stopwords) 🙈. Specify a whole percentage and a number, such as "80 256". This omits words that occur in over 80% of the files and appear in over 256 files 🧮. Comment it out to turn off auto-stopwording 💤.
IgnoreLimit 50 1000Swish-e has to do extra processing to adjust the entire index when this feature is used 😩. It's recommended that, instead of using this feature, you decide which words are stopwords yourself and add them to IngoreWords in your configuration file 🧠. To do this, use IgnoreLimit one time and note the stop words that get found while indexing. Add that list to IgnoreWords, and then remove IgnoreLimit from the configuration file 🫶.
IgnoreMetaTags list of names
IgnoreMetaTags defines a list of metatags to ignore while indexing XML files (and HTML files if using libxml2 for parsing HTML) 🙅♀️. All text within those tags gets ignored, both for indexing ("MetaNames") and for properties ("PropertyNames"). If you want to still parse properties but not index the text, see "UndefinedMetaTags" 🎀.
This option is useful for avoiding indexing specific data from a file 🔐. For example:
<person>
<first_name>
William
</first_name> <last_name>
Shakespeare
</last_name> <updated_date>
April 25, 1999
</updated_date>
</person>In the above example, you might not want to index the updated date, and therefore prevent finding this record by searching:
-w 'person=(April)'This is solved by:
IgnoreMetaTags updated_dateSee also "UndefinedMetaTags" 👑.
🧪 IgnoreNumberChars list of characters
🧪 Experimental Feature — this one is still in its experimental era, so it may change in future versions. One possible change is to use regular expressions instead. 🙈
okay so this one is giving number filter. ✨ You list a set of characters that describe a number, and if a word is made up of only those characters, Swish-e will skip indexing it. The characters you list have to already be part of your WordCharacters settings, because the "word" being checked is one Swish-e would otherwise index anyway. 💅
For example:
IgnoreNumberChars 0123456789$.,With that set, Swish-e would not index any of these:
123
123,456.78
$123.45🙈 Bestie, careful with hex. You might be tempted to skip hex numbers with:IgnoreNumberChars 0123456789abcdefThat won't index0D31, but it also won't index the word "bad". 💀
💬 IndexComments [NO⎪yes]
this option lets the user decide whether the contents of HTML comments get indexed. 🫶 The default is no, so set it to yes if you actually need comment indexing.
IndexComments yesNote: this is a change from the default behaviour before version 2.2, so keep that in mind if you're upgrading old configs. 🌸
🔁 TranslateCharacters [*string1 string2*⎪:ascii7:]
the TranslateCharacters directive maps the characters in string1 to the characters in string2. It's like a makeover for your characters. 💅
# This will index a_b as a-b and ámo as amo
TranslateCharacters _á -aTranslateCharacters :ascii7: is a predefined set that turns eight bit characters into ascii7 characters. So with the :ascii7: rule, "Ääç" becomes "aac", which means searching "Çelik", "çelik" or "celik" will all match the same word. ✨ Iconic. 🦋
Here's the timing tea ☕: translation happens early in the indexing process, after HTML entities are converted but before the input gets split into words based on WordCharacters. So the characters you're translating from don't need to be listed in word characters. 🧚 The same character translations also happen when searching. 🔥
🫧 BumpPositionCounterCharacters string
when Swish-e indexes, it gives each word a position number, and that's what makes phrase searching work. 🌈 Sometimes you don't want phrases matching, though. The BumpPositionCounterCharacters directive lets you name a set of characters that, when they show up in the text, bump the word position up. That basically stops phrase matches from crossing that character. 🙅♀️
For example, if you have a tag like this:
<subjects>
computer programming ⎪ apple computers
</subjects>You might want to stop "programming apple" from matching in that meta name, so you'd write:
BumpPositionCounterCharacters ⎪There is no default, and you can list a whole string of characters. 💜
🪄 DontBumpPositionOnEndTags list of names & DontBumpPositionOnStartTags list of names
okay this one's a little plot twist. 🎀 Metatags are usually separate data fields, so the word position counter gets bumped automatically between metatags. It happens when a start tag is found and when an end tag is found too. This stops a phrase from matching across more than one metaname. 🙈
DontBumpPositionOnEndTags and DontBumpPositionOnStartTags turn that feature off for the metanames you list. 🫶
For example, say you have this document:
<person>
<first_name>
William
</first_name>
<last_name>
Shakespeare
</last_name>
<updated_date>
April 25, 1999
</updated_date>
</person>In the configuration file, you'd write:
DontBumpPositionOnEndTags first_name
DontBumpPositionOnStartTags last_nameWith that config, this phrase search works:
-w 'person=("william shakespeare")'but this one will fail:
-w 'person=("shakespeare april")'📁 Directives for the File Access method only
some directives do different jobs depending on where your documents come from. ✨ These ones are only valid when you're using the File system method of indexing. 🗂️
📄 IndexOnly list of file suffixes
this directive sets which file suffixes (extensions) are allowed while indexing. 🌷 The default is to index all files found under IndexDir.
# Only index .html .htm and .q files
IndexOnly .html .htm .qQuick heads-up: IndexOnly checks that a file name ends in the characters listed. It does not check "extensions". 🧐 It's tested right before FileRules gets processed.
🔗 FollowSymLinks [yes⎪NO]
put yes to follow symbolic links while indexing, or no if you don't. 💖 The default is no.
FollowSymLinks no
FollowSymLinks yesHere's the tea: when this is set to no, extra stat(2) system calls have to be made for each file. ☕ If you've got a large number of files, setting it to yes might give you a small cut in indexing time. ⭐
See also the -l switch in SWISH-RUN. 🦋
🚦 FileRules [type] [contains⎪is⎪regex] regular expression & FileMatch [type] [contains⎪is⎪regex] regular expression
FileRules and FileMatch are how you exclude and include files and directories when indexing, respectively. 🫧 By default Swish-e indexes all files and recurses into all directories (but see also FollowSymLinks), so you'll usually only reach for FileRules to exclude stuff. FileMatch comes in clutch in a few cases, like overriding what IndexOnly did. Some examples are below. 👑
Heads up: apart from FileRules title ..., this feature is only available for the file access method (-S fs), which is the default indexing mode. 🌈 Also, any pathname changes made with ReplaceRules happen after the FileRules check. (It's unlikely you'd exclude files with FileRules based on text you added with ReplaceRules!) 🙃
The regular expression is a C regex.h extended regular expression. 🧚 You can supply more than one regular expression per line, or use separate directives. Putting the word not in front of the regular expression negates the match. 🔥
The regular expression gets compared against the type described below.
For historical reasons, you can specify contains or is. is just forces the regular expression to match at the start and end of the string, which it does by internally prepending ^ and appending $ to the regular expression. 💅
The regex option needs delimiter characters:
FileRules title regex /^private/iThe only real perk of regex is if you want case insensitive matches, or you just like your regular expressions to look like Perl regular expressions. 💜 You must use matching delimiters. (), {}, and [] aren't currently supported, and the only reason is laziness. 💀
Use quotes (" or ') around a pattern if it contains any white space. Also, the backslash becomes the escape character inside quotes, so watch out. ⚠️
These all generate the same regular expressions:
FileRules title is hello
FileRules title contains ^hello$
FileRules title regex /^hello$/These all need quotes because of the space character:
FileRules title is "hello there"
FileRules title contains "^hello there$"
FileRules title regex "!^hello there$!"These show how the backslash has to be doubled inside quotes. Swish-e turns a double-backslash into a single backslash, and then passes that single one on to the regular expression compiler. 🤓
FileRules filename regex /\.pdf/
FileRules filename regex "/\\.pdf/"
FileRules filename regex !hello\\there! # need double for real backslash
FileRules filename regex "!hello\\\\there!" # need double-double inside of quotesMatching Types 🎀
The following types of match strings may be supplied:
FileRules pathname
FileRules dirname
FileRules filename
FileRules directory
FileRules title
FileMatch pathname
FileMatch filename
FileMatch dirname
FileMatch directorypathname matches the regular expression against the current pathname. Whether it's absolute or not depends on what you gave to IndexDir. 🗺️
Example:
# Don't index paths that contain private or hidden
FileRules pathname contains (private⎪hidden)
# Same thing
FileRules pathname regex /(private⎪hidden)/
# Don't index exe files
FileRules pathname contains \.exe$dirname and filename split the path name at the last delimiter character into a directory name and a file name. Then those get compared against your patterns. Directory names don't have a trailing slash, and all path names use the forward slash as a delimiter within Swish-e. 🦋
Example:
# Same as last example - don't index *.exe files.
FileRules filename contains \.exe$
# Don't index any file called test.html files
FileRules filename contains ^test\.html$
# Same thing
FileRules filename is test\.html
# Don't index any directories that contain "old" (/usr/local/myold/docs)
FileRules dirname contains old
# Don't index any directories that contain the path segment "old" (/usr/local/old/foo)
FileRules dirname contains /old/
# Index only .htm, .html, plus any all-digit file names
IndexOnly .htm .html
FileMatch filename contains ^\d+$
# Same as previous, but maybe a little slower
FileRules filename regex not !\.(htm⎪html)$!
FileMatch filename contains ^\d+$Swish-e checks these settings in the order of pathname, dirname, and filename. In general, FileMatch patterns get checked before FileRules. ✨ This means you can exclude most files with FileRules and still let in a few special cases with FileMatch. 🫶 For example:
# Exclude all files of .exe, .bin, and .bat
FileRules filename contains \.(exe⎪bin⎪bat)$
# But, let these two in
FileMatch filename is baseball\.bat incoming_mail\.bin
# Same, but as a single pattern
FileMatch filename is (baseball\.bat⎪incoming_mail\.bin)The "directory" type is kinda unique 🦋 When Swish-e recurses into a directory, it checks every file in that directory against the pattern, then decides if the entire directory gets indexed (or recursed) or not. ngl that's giving "one bad apple spoils the bunch" energy 🍎 Remember you're matching against file names in a directory, and some of those names might be directory names too 📂
A "FileRules directory" match makes Swish-e ignore all files and sub-directories in the current directory. Total main character energy for exclusion 💅
⚠️ Warning! A match with "FileMatch directory" says to index everything in the current directory and ignore any FileRules for this directory. Handle with care, bestie 😭
Example:
# Don't index any directories (and sub directories) that contain
# a file (or sub-directory) called "index.skip"
FileRules directory contains ^index\.skip$
# Don't index directories that contain a .htaccess file.
FileRules directory contains ^\.htaccessNote: While processing directories, Swish-e will skip any files or directories that begin with a dot (.) 🙈 If you do want dot-files or dot-directories indexed, name them with IndexDir or -i and they'll get in ✨
"title" checks for a pattern match in an HTML title 🎀
Example:
FileRules title contains construction example pointers
# This example says to ignore case
FileRules title regex "/^Internal document/i"Note: FileRules title works for any input method (fs, prog, or http) that gets parsed as HTML, and only where a title was actually found in the document 📄
Okay so if all this feels a bit confusing, don't worry, it's a lot 😭 Here's the order Swish-e processes a directory in, step by step 💖
First, the directory name gets checked:
FileRules dirname - reject entire directory if matchesNext, the directory gets scanned and each file name (which might be a sub-directory name) gets checked:
FileRules directory - reject entire dir if *any* files match
FileMatch directory - accept entire dir if *any* files matchThen, unless "FileMatch directory" matched, each file gets tested with FileMatch. A match says to index the file without further testing, which means it overrides FileRules and IndexOnly 🔥
FileMatch pathname \
FileMatch dirname - file is accepted if any match
FileMatch filename /Otherwise:
IndexOnly - file is checked for the correct file extension
FileRules pathname \
FileRules dirname - file is rejected if any match
FileRules filename /Finally, the file gets indexed 🎉
Files (not directories) listed with IndexDir or -i get processed in a similar way 🧚 First up, FileMatch:
FileMatch pathname \
FileMatch dirname - file is accepted if any match
FileMatch filename /Otherwise, the file gets rejected if it doesn't have the correct extension or if a FileRules matches:
IndexOnly - file is checked for the correct file extension
FileRules pathname \
FileRules dirname - file is rejected if any match
FileRules filename /Note: If things aren't indexing the way you expect, make a directory with some test files and use the -T regex trace option to see how file names are checked 🔍 Start with very simple tests, no cap!
✨ Directives for the HTTP Access Method Only ✨
The HTTP Access method is turned on with the -S http switch when indexing 🌐 It works by running a Perl program called SwishSpider, which fetches documents from a web server.
Only text files (content-type of "text/*") get indexed with the HTTP Access Method. Other document types (e.g. PDF or MSWord) may be indexed as well 📑 SwishSpider will try to use the SWISH::Filter module (included with the Swish-e distribution) to convert documents into a format Swish-e can index 🔧
Note: The -S prog method of spidering (using spider.pl) can be a replacement for the -S http method. It offers more configuration options and better spidering speed 🚀
These directives below are available when using the HTTP Access Method of indexing 💜
🌸 MaxDepth integer
MaxDepth defines how many links the spider should follow before stopping. A value of 0 tells the spider to traverse all links. The default is MaxDepth 0.
MaxDepth 5Note: The default was changed from 5 to 0 in release 2.4.0 🦋
🫶 Delay seconds
This is the number of seconds to wait between issuing requests to a server. It's a super friendly way to spider remote sites 🥰 The default is 5 seconds.
Delay 1Note: The default was changed from 60 to 5 seconds in release 2.4.0 ⭐
🍓 TmpDir path
This is the location of a writable temp directory on your system. The HTTP access method tells the Perl helper to put its files here, and the -e switch makes Swish-e use this directory while indexing. There is no default 🙅♀️
TmpDir /tmp/swish⚠️ If this directory doesn't exist or isn't writable, Swish-e will fail with an error during indexing 💀
Note, the environment variables TMPDIR, TMP, and TEMP (in that order) will override this setting 🌈
💅 SpiderDirectory path
This is the location of the Perl helper script called swishspider. If you use a relative directory, it's relative to your directory when you run Swish-e, not to the directory that Swish-e is in 🗺️ The default is the location swishspider was installed. Normally you don't need to set this 🙌
SpiderDirectory /usr/local/swish👑 EquivalentServer server alias
Okay so the same site can be referred to by different names all the time 🙃 A common example is that http://www.some-server.com and http://some-server.com are literally the same place. Each line should have a list of all the method/names that should be considered equivalent. You can use multiple EquivalentServer directives, and each one defines its own set of equivalent servers 💖
EquivalentServer http://library.berkeley.edu http://www.lib.berkeley.edu
EquivalentServer http://sunsite.berkeley.edu:2000 http://sunsite.berkeley.edu🧚 Directives for the prog Access Method Only 🧚
This section covers the directives that only work with the "prog" document source feature of Swish-e 💫 The prog access method runs an external program that "feeds" documents to Swish-e, which means you can index and filter documents from literally any source 🌍
See prog - general purpose access method in the SWISH-RUN man page for more information 📖
A bunch of example programs for the prog access method live in the prog-bin directory. Check those out if you have questions about writing a prog input program 🎀
✨ SwishProgParameters list of parameters
This is a list of parameters that gets sent to the external program when you run with the "prog" document source method. It's the easiest way to pass settings from the Swish-e configuration file to the external program 🫶
SwishProgParameters /path/to/config hello there
IndexDir /path/to/program.plThen running:
swish-e -c config -S progSwish-e will execute /path/to/program.pl and pass /path/to/config hello there to the program as three command line arguments 🔥
For example, the spider.pl program (included in the prog-bin directory) uses SwishProgParameters to specify which file to read for configuration information 🧠
SwishProgParameters spider.config
IndexDir ./spider.plThe spider.pl program also has a default action, so you can skip the configuration file entirely 🥳
SwishProgParameters default http://www.swishe.org/ http://some.other.site/
IndexDir ./spider.plAnd the spider program will use default settings for spidering those sites 🕷️
Swish-e can also read documents from standard input, so another way to run an external program with parameters looks like this 🌷
./spider.pl spider.conf ⎪ ./swish-e -S prog -i stdin🎀 Notes when using MS Windows 🎀
You should use unix style path separators to specify your external program 🪟 Swish will convert forward slashes to backslashes before calling the external program. This only applies to the program name specified with IndexDir or the -i command line option.
Also, Swish-e will make sure the program you specify actually exists, which means you need to use the full name of the program 🫠
For example, to run the perl spider program spider.pl, you'd need a Swish-e configuration file like this:
IndexDir e:/perl/bin/perl.exe
SwishProgParameters prog-bin/spider.pl default http://swish-e.orgThen run indexing with the command:
swish-e -c swish.cfg -S prog -v 9The IndexDir command tells Swish-e the name of the program to run 🏃♀️ Under unix you can just specify the name of the script, since unix will figure out the program from the first line of the script 🐧
The SwishProgParameters are the parameters passed to the program specified by IndexDir (perl.exe in this case). The first parameter is the perl script to run (prog-bin/spider.pl), and perl passes the rest of the parameters directly to the perl script. The second parameter, default, tells the spider.pl program to use default settings for spidering (or you could specify a spider config file. See perldoc spider.pl for details). Lastly, the URL gets passed into the spider program 💜
🦋 Document Filter Directives 🦋
Internally, Swish-e only knows how to parse text, HTML, and XML documents 📚 With filters you can index other types of documents too. For example, if all your web pages are in gzip format, a filter can uncompress them on the fly while indexing 🗜️
You may wish to read the Swish-e FAQ question on filtering before continuing here. How Do I filter documents? 🤔
There are two suggested methods for filtering 💅
🌈 Filtering with SWISH::Filter 🌈
The Swish-e distribution includes a Perl module called SWISH::Filter and individual filters located in the filters directory 🧁 This system uses plug-in filters to extend the types of documents Swish-e can index. The plug-in filters don't actually do the filtering themselves. They just provide a standard interface for accessing programs that can filter or convert documents 🔌 The programs that do the filtering are not part of the Swish-e distribution, so they must be downloaded and installed separately 📦
The advantage of this method? New filtering methods can be installed easily 🙌
This system is designed to work with the -S http and -prog methods, but it can also vibe with the "FileFilter" feature and the -S fs indexing method 🦋. Peep this script for an example 🌸:
$prefix/share/doc/swish-e/examples/filter-bin/swish_filter.plAlso, the filters/README file has more info, so go read it when you get a sec 📖✨
🎀 Filtering with the FileFilter feature 🎀
A filter is an external program that Swish-e runs while it's processing a document of a certain type 💅. Swish-e runs the filter program for every file that matches the file suffix (extension) set in the FileFilter or FileFilterMatch directives. FileFilterMatch matches using regular expressions, and we'll get to that below 🌈.
Filters can be used with any type of input method (aka -S fs, -S http, or -S prog). But because…
Swish-e calls the external program and passes these as default arguments, so bestie, take note 📝:
$0— the name of the filter program$1— the physical path name of the file to read. This might be a temporary file location if you're indexing by the http method 🗂️$2— when you're indexing under the file system, this is the same as$1(the path to the source file). But when you're indexing under the http method, this is the URL of the source document 🔗
Swish-e can also pass other parameters to the filter program. You define these using the FileFilter or FileFilterMatch directives. See Filter Options below 💖.
The filter program has to open the file, process its contents, and hand it back to Swish-e by printing to STDOUT 🖨️.
⚠️ Heads up, bestie: if your external program is a perl or shell script, this can add a lot of time to the indexing process 😭. If you have tons of files to filter, you should think about writing your filter in C instead of a shell or perl script, or using the "prog" Access Method along with SWISH::Filter 💀.
FilterDir
FilterDir *path-to-directory*Deprecated. Yeah, this one is on its way out 🥀.
This is the path to a directory where your filter programs live. Swish-e looks in this directory to find the filter named in the FileFilter directive 📂.
You don't need this if the filter program can be found in your system's path ✨. Even if your filter isn't in your system's path, you can still give the full path to the filter in the FileFilter or FileFilterMatch directives 🧚.
Example:
FilterDir /usr/local/swish/filtersFileFilter
FileFilter *suffix* "filter-prog" ["filter-options"]This maps a file suffix (extension) to a filter program 🦋. If filter-prog starts with a directory delimiter (aka an absolute path), Swish-e skips the FilterDir settings and uses the given filter-prog path directly 🎯.
On systems with a working fork(2) system call, the filter program gets run by forking swish and then executing the filter. This means the shell isn't used to run the filter, and no arguments go through the shell 🙌.
On other systems (like Windows), the arguments are double-quoted and popen(3) is used to run the program. This does pass arguments through the shell, which may be a security concern depending on what the shell is able to do 🚨.
Filter options:
Filter options are a string passed as arguments to the filter-prog 🎀. Filter options can contain variables, which Swish-e replaces for you ✨. If you leave out filter-options, Swish-e will use the default parameters for the options listed above 💅.
Default:
%p %PWhich means you're passing "workfile path" and "documentfile path" to the filter 🌸.
Variables in filter options:
%%— a literal percent sign (%) 💅%P— full document pathname (e.g. URL, or path on filesystem) 🔗%p— full pathname to work file (maybe a tmpfile or the real document path on filesystem) 🗃️%F— filename stripped from full document pathname 📄%f— filename stripped from "work" pathname 📝%D— directoryname stripped from full document pathname 📁%d— directoryname stripped from full "work" pathname 🌷
Examples of strings passed:
%P = document pathname: http://myserver/path1/mydoc.txt
%p = work pathname: /tmp/tmp.1234.mydoc.txt
%F = mydoc.txt
%f = tmp.1234.mydoc.txt
%D = http://myserver/path1
%d = /tmpNotes when using MS Windows
Windows uses double quotes to escape shell metacharacters 🪟. So if you need quotes, wrap the entire option string in single quotes 💫.
FileFiler .mydoc mydocfilter.exe '--title "text with spaces"'You can specify the filter program with forward slashes (unix style), and Swish will convert the slashes to backslashes before running your program 🔁.
FileFilter .mydoc c:/some/path/mydocfilter.exe '-d "%d" -example -url "%P" "%f"'Examples of filters:
FileFilter .doc /usr/local/bin/catdoc "-s8859-1 -d8859-1 %p"
FileFilter .pdf pdftotext "%p -"
FileFilter .html.gz gzip "-c %p"
FileFilter .mydoc "/some/path/mydocfilter" "-d %d -example -url %P %f"The examples above are running a binary filter program 🏋️. For more complicated filtering needs, you can use a scripting language like Perl or a shell script 📜. Here are some examples of calling a shell script and a perl script 🐚:
FileFilter .pdf pdf2html.sh
FileFilter .ps ghostscript-filter.pl🐢 Using a scripting language (or any language with a large startup cost) can greatly increase the indexing time. For small indexing jobs this might not be a big deal, but for large collections of files that need processing by a scripting language, you may be better off using the "-S prog" access method, where the script only gets compiled once instead of once per document 💀.
Filters are probably easier to write than a "-S prog" program 🥰. Which one you pick depends on your requirements 💅. Examples of filter scripts can be found in the filter-bin directory, and examples of "-S prog" programs can be found in the prog-bin directory 🧚♀️.
FileFilterMatch
FileFilterMatch *filter-prog* *filter-options* *regex* [*regex* ...]This is similar to "FileMatch", except it uses regular expressions to match against the file name 🔍. filter-prog is the path to the program. Unlike "FileFilter", this does not use the "FilterDir" option 🙅. Also unlike "FileFilter", you must specify the filter-options 🫶.
Examples:
FileFilterMatch ./pdftotext "%p -" /\.pdf$/Heads up, this will also match a file literally named ".pdf" 🙈. So you might want something that requires a filename with more than just an extension. For example:
FileFilterMatch ./pdftotext "%p -" /.\.pdf$/To specify more than one extension:
FileFilterMatch ./check_title.pl "%p" /\.html$/ /\.htm$/Or a few ways to do the same thing:
FileFilterMatch ./check_title.pl %p /\.(html⎪html)$/
FileFilterMatch ./check_title.pl %p /\.html?$/And to ignore case:
FileFilterMatch ./check_title.pl %p /\.html?$/iYou can also put "not" in front of an expression to negate the regular expressions that follow it 🙃. For example, to match files that do not have an extension:
FileFilterMatch ./convert "%p %P" not /\..+$/📜 Document Info 📜
$Id: SWISH-CONFIG.pod 1846 2006-10-20 20:18:30Z whmoseley $That's the whole page, bestie 🫶💖 Version 2.4.7, dated 2009-04-04, SWISH-CONFIG(1) 🌈✨
📦 where this page comes from
- debian bookworm: swish-e 2.4.7-6.1+b2
- debian trixie: swish-e 2.4.7-6.3
- ubuntu jammy: swish-e 2.4.7-6.1build1
- ubuntu noble: swish-e 2.4.7-6.2build3
- ubuntu questing: swish-e 2.4.7-7
- ubuntu resolute: swish-e 2.4.7-7.1
- ubuntu stonking: swish-e 2.4.7-7.2build1
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms