index++(1)
okay bestie, this is the index++(1) page from the General Commands Manual and it's giving absolute Unix royalty. ๐ buckle up, we're indexing everything.
โจ NAME โจ
index++โ SWISH++ indexer. that's the whole vibe. ๐
โจ SYNOPSIS โจ
index++ [ options ] directory... file...
โจ DESCRIPTION โจ
okay so index++ is the SWISH++ file indexer. it indexes the files you specify and the files inside the directories you specify. by default, files in subdirectories of those specified directories also get indexed โ unless you pass either the -r or --no-recurse option, or the RecurseSubdirs variable is set. ๐โจ
files are indexed either only if their filename matches one of the patterns in the set you gave with either the -e or --pattern option or the IncludeFile variable (unless standard input is used โ next paragraph, bestie) or if the file is not in the set you gave with either the -E or --no-pattern option or the ExcludeFile variable. yes, it's giving "either this or that" energy. ๐ซถ
if there's a single filename of -, then the list of directories and files to index is instead taken from standard input, one per line. in this case, filename patterns of files to index don't need to be specified explicitly: all files, regardless of whether they match a pattern (unless they're in the set not to index specified with either the -E or --no-pattern option or the ExcludeFile variable), are indexed. i.e., index++ assumes you know what you're doing when you specify filenames this way. periodt. ๐
in any case, care must be taken not to specify files or subdirectories inside directories that are also specified. since directories are recursively indexed by default (unless either the -r or --no-recurse option or the RecurseSubdirs variable is given), explicitly specifying a subdirectory or file in a directory that is also specified will result in those files being indexed more than once. not the duplicate era we want. ๐ญ
๐ Character Mapping ๐
characters in the ISO 8859-1 (Latin 1) character set are mapped to their closest ASCII equivalent before further examination and indexing. (individual indexing modules may also do their own character mapping.) it's giving glow-up before the main event. โจ
๐ Word Determination ๐
stop words โ words that occur too frequently or have no information content โ are not indexed. (there's a default built-in set of a few hundred such English words.) additionally, several heuristics are used to determine which words should not be indexed. ๐ง
first, a word is checked to see if it looks like an acronym. a word is considered an acronym only if it starts with a capital letter and is composed exclusively of capital letters, digits, and punctuation symbols, e.g., AT&T. if a word looks like an acronym, it is indexed and no further checks are done. iconic behavior. ๐
second, there are several other checks that are applied. a word is not indexed if it:
- is less than
Word_Min_Sizeletters. (default is 4.) - contains less than
Word_Min_Vowelsvowels. (default is 1.) - contains more than
Word_Max_Consec_Sameof the same character consecutively (not including digits). (default is 2.) - contains more than
Word_Max_Consec_Consonantsconsecutive consonants. (default is 5.) - contains more than
Word_Max_Consec_Vowelsconsecutive vowels. (default is 4.) - contains more than
Word_Max_Consec_Punctsconsecutive punctuation characters. (default is 1.)
๐ Filters ๐
via the FilterFile configuration file variable, files matching particular patterns can be filtered prior to indexing. via the FilterAttachment configuration file variable, e-mail attachments whose MIME types match particular patterns can be filtered prior to indexing. (see FILTERS in swish++.conf(5).) filtered before the index even sees them โ lowkey protective. ๐ก๏ธ
๐ Incremental Indexing ๐
in order to add words from new documents to an existing index++, either the entire set of documents can be reindexed or the new documents alone can be incrementally indexed. in many cases, reindexing everything is sufficient since index++ is really fast. for a very large document set, however, this may use too many resources. ๐ฎโ๐จ
however, there is a pitfall for incremental indexing: if any of the -f, --word-files, -p, or --word-percent options or WordFilesMax or WordPercentMax variables are used, then words that are too frequent are discarded. if new documents are added containing very few of those words, then they could no longer be too frequent. however, there is no way to get them back since they were discarded. ๐
the way around this problem is not to discard any words by specifying 101%. however, because no words are discarded, the size of the index file will be larger, perhaps significantly so. bigger index, bigger feelings. ๐ซ
it is possible that, in practice, the loss of words may not be that important โ especially if new documents are very similar to old documents and that words that were too frequent in the old set would also be too frequent in new set.
another way around this problem is to do periodic full indexing. self-care, but for your index. ๐งโโ๏ธ
โจ INDEXING MODULES โจ
index++ is written in a modular fashion where different types of files have different indexing modules. currently, there are 7 modules: Text (plain text), HTML (HTML and XHTML), ID3 (ID3 tags found in MP3 files), LaTeX, Mail (RFC 822 and Usenet News), Manual (Unix manual pages in nroff(1) with man(7) macros), and RTF (Rich Text Format). squad goals. ๐ฏโโ๏ธ
๐ Text Module ๐
this module simply indexes plain text files performing character mapping and word determination as has already been described. simple. clean. no drama. โจ
๐ HTML and XHTML Module ๐
additional processing is done for HTML and XHTML files. the additional processing is:
- character and numeric (decimal and hexadecimal) entity references are converted to their ASCII character equivalents before further examination and indexing. for example,
résumébecomesresumebefore indexing. glow-up complete. ๐ - if a matched set of
<TITLE>...</TITLE>tags is found within the firstTitleLineslines of the file (default is 12), then the text between the tags is stored in the generated index file as the file's title rather than the file's name. (every non-space whitespace character in the title is converted to a space; leading and trailing spaces are removed.) title drop, but make it searchable. ๐ค - if an HTML or XHTML element contains a
CLASSattribute whose value is among the set of class names specified as those not to index (via one or more of either the-Cor--no-classoption or theExcludeClassvariable), then all the text up to the tag that ends the element will not be indexed. for an element that has an optional end tag, "the tag that ends the element" is either the element's end tag or a tag of another element that implicitly ends it; for an element that does not have an end tag, "the tag that ends the element" is the element's start tag. (see the EXAMPLES.) all elements from the HTML 4.0 specification (including deprecated elements), Ruby elements, plus common, browser-specific elements are recognized; unrecognized elements are ignored. (see the-Hor--dump-htmloption.) - if an HTML or XHTML element contains a
TITLEattribute, then the words specified as the value of theTITLEattribute are indexed. attribute said "index me" and we did. ๐ซถ - if an
AREA,IMG, orINPUTelement contains anALTattribute, then the words specified as the value of theALTattribute are indexed. alt text really said "i'm the moment." ๐ธ - if a
METAelement contains both aNAMEandCONTENTattribute, then the words specified as the value of theCONTENTattribute are indexed associated with the meta name specified as the value of theNAMEattribute. (however, if either the-Aor--no-assoc-metaoptions or theAssociateMetavariable is specified, then the words specified as the value of theCONTENTattribute are still indexed, but not associated with the meta name.) (see also the-m,--meta,-M, and--no-metaoptions or theIncludeMetaorExcludeMetavariables.) meta names can later be queried against specifically usingsearch++(1). meta, but make it fashion. ๐ - if a
TABLEelement contains aSUMMARYattribute, then the words specified as the value of theSUMMARYattribute are indexed. table summary said "don't skip me." ๐ - if an
OBJECTelement contains aSTANDBYattribute, then the words specified as the value of theSTANDBYattribute are indexed. standby text gets its moment. โจ - all other HTML or XHTML tags and comments (anything between
<and>characters) are discarded. bye bestie. ๐
in compliance with the HTML specification, any one of no quotes, single quotes, or double quotes may be used to contain attribute values and attributes can appear in any order. values containing whitespace, however, must be quoted. the specification is vague as to whether whitespace surrounding the = is legal, but index++ allows it. flexible queen behavior. ๐
๐ ID3 Module ๐
ID3 tags are used to store audio meta information for MP3 files (generally). since audio files contain mostly binary information, only the ID3 tag text fields are indexed. ID3 tag versions 1.x and 2.x (through 2.4) are supported (except for encrypted frames). if a file contains both 1.x and 2.x tags, only the 2.x tag is indexed. the processing done for files containing an ID3 tag is:
- if a title field is found, then the value of the title is stored in the generated index file as the file's title rather than the file's name. (every non-space whitespace character in the title is converted to a space; leading and trailing spaces are removed.) title field said main character energy. ๐ถ
- words that are the value of fields are indexed associated with the field name as a meta name. (however, if either the
-Aor--no-assoc-metaoptions or theAssociateMetavariable is specified, then the words specified as the value of the field are still indexed, but not associated with the field.) (see also the-m,--meta,-M, and--no-metaoptions or theIncludeMetaorExcludeMetavariables.) meta names can later be queried against specifically usingsearch++(1).
for ID3v1.x, the recommended fields to be indexed are: album, artist, comments, genre, and title.
for ID3v2.2, the recommended text fields (with reassignments) to be indexed are: com=comments, tal=album, tcm=composer, tco=genre, tcr=copyright, ten=encoder, txt=lyricist, tt1=content, tt2=title, tt3=subtitle, ipl=musicians, tot=original-title, tol=original-lyricist, toa=original-artist, tp1=artist, tp2=performers, tp3=conductor, tpb=publisher, txx=user, slt=lyrics, and ult=lyrics.
for ID3v2.4, the recommended text fields (with reassignments) to be indexed are: comm=comments, talb=album, tcom=composer, tcon=genre, tcop=copyright, tenc=encoder, text=lyricist, tipl=people, tit1=content, tit2=title, tit3=subtitle, tmcl=musicians, tmoo=mood, toal=original-title, toly=original-lyricist, tope=original-artist, town=owner, tpe1=artist, tpe2=performers, tpe3=conductor, tpub=publisher, tsst=set-subtitle, txxx=user, user=terms, sylt=lyrics, and uslt=lyrics.
ID3v2.3 is the same as 2.4 except replace tmcl=musicians with ipls=musicians.
- For
ID3v2.x, any text fields that are compressed get uncompressed before indexing. we're not indexing the zip file bestie, we're indexing the actual words ๐ - Also for
ID3v2.x: Unicode text encoded in eitherUTF-8orUTF-16(big-endian or little-endian, both valid, we don't play favourites) gets decoded before indexing. no cursed garbled bytes in your index, periodt โจ
๐ LaTeX Module ๐
LaTeX files get extra special treatment on top of the usual stuff. The tea is:
- If a
\titlecommand is found within the firstTitleLineslines of the file (default is12), then the value of that title is what gets stored in the generated index file as the file's title โ not the file's name. ๐ซถ (Every non-space whitespace character in the title gets converted to a space, and leading and trailing spaces get removed.)
๐ Mail Module ๐
Mail and news files also get extra processing. Here's the whole rundown, bestie, no skipping:
- If a
Subjectheader is found within the firstTitleLineslines of the file (default is12), then the value of that subject is what gets stored in the generated index file as the file's title, instead of the file's name. ๐ฆ (Every non-space whitespace character in the title is converted to a space; leading and trailing spaces are removed.) - Words that are the value of a header get indexed associated with the header name as a meta name. So like, the words live under the header's name, iykyk.
โ ๏ธ BUT โ if either the-Aor--no-assoc-metaoptions or theAssociateMetavariable is specified, then the words specified as the value of the header are still indexed, they're just not associated with the header. still in the index, just no longer holding hands with its header.
(See also the -m, --meta, -M, and --no-meta options or the IncludeMeta or ExcludeMeta variables.) Meta names can later be queried against specifically using search++(1). ๐
The recommended headers to be indexed are: Bcc, Cc, Comments, Content-Description, From, Keywords, Newsgroups, Resent-To, Subject, and To.
- MIME attachments get indexed. yes, the attachments. all of them. ๐
- Text that's in the
text/enrichedcontent type gets converted to plain text before indexing. - Text that's encoded as either
quoted-printableorbase-64gets decoded before indexing. - Unicode text encoded in either the
UTF-7orUTF-8character set gets decoded before indexing. - Text in vCards gets indexed in a way where the values of types (fields) are associated with those types as meta names. ๐ชช
โ ๏ธ Same caveat as before: if either the-Aor--no-assoc-metaoptions or theAssociateMetavariable is specified, then the words specified as the value of types are still indexed, but not associated with the types.
The recommended vCard types (with reassignments) to be indexed are: adr=address, categories, class, label=address, fn=name, nickname, note, org, role, and title.
Okay so real talk ๐ซถ โ indexing mail and news files is most effective only when there's exactly one message per file. Usenet news files are usually like that, but mail files? not so much. Mail files, e.g., mailboxes, are usually made up of multiple messages. ๐ญ Such files would need to be split up into files of individual messages prior to indexing, because there's literally no point in indexing a single mailbox: every single search result would return a rank of 100 for the same file. like, every. single. one. 100, 100, 100. That's why the splitmail++(1) utility is included in the SWISH++ distribution. ๐ง
๐ Manual Module ๐
Unix manual page files get extra processing too. Here's what goes down:
- If a
NAMEsection heading macro (.SH) is found within the firstTitleLineslines of the file (default is12), then the contents of the next line get stored in the generated index file as the file's title โ instead of the file's name. ๐ธ (Every non-space whitespace character in the title is converted to a space; leading and trailing spaces, plus backslash sequences such as\f2, are removed.) - Words that are in a section get indexed associated with the name of that section as a meta name.
โ ๏ธ You already know what's coming: if either the-Aor--no-assoc-metaoptions or theAssociateMetavariable is specified, then the words in a section are still indexed, but not associated with the section heading.
Spaces in multi-word section headings get converted to dashes โ for example, see also'' becomes see-also'' as a meta name. ๐ (See also the -m, --meta, -M, and --no-meta options or the IncludeMeta or ExcludeMeta variables.) Meta names can later be queried against specifically using search++(1).
The recommended sections to be indexed are: AUTHOR, BUGS, CAVEATS, DESCRIPTION, DIAGNOSTICS, ENVIRONMENT, ERRORS, EXAMPLES, EXIT-STATUS, FILES, HISTORY, NAME, NOTES, OPTIONS, RETURN-VALUE, SEE-ALSO, SYNOPSIS, and WARNINGS. ๐
๐ RTF Module ๐
This module is the low-effort icon of the group: it simply indexes rich text format files without all the formatting commands. no \rtf nonsense in your search results, just the words. slay. ๐
๐ OPTIONS ๐
okay bestie so options are giving two flavors: short ones start with a single - and long ones start with a --. and yes, a lone - or -- all by itself is the "we're done with options now" signal โ explicit vibes only. you can mix short and long however you want.
long option names can be abbreviated as long as the abbreviation is unambiguous โ like, no two options fighting over the same prefix, iykyk.
short option arguments (the rules) ๐
for a short option that takes an argument, the argument is either the rest of the characters in that same option (if there are any), or, if not, it comes from the next option โ unless that next thing starts with a -.
short options with no arguments can be grouped together (but the last one in the group is allowed to take an argument), so like -lrv4 is literally the same as -l -r -v4. iconic.
long option arguments (the rules) ๐ธ
for a long option that takes an argument, the argument is either the characters after a =, if there are any, or, if not, it comes from the next option โ unless that next thing starts with a -.
-?,--helpโ prints the usage (a.k.a. the "help" message) and then exits. your lil emergency exit ๐-A,--no-assoc-metaโ do not associate words with meta names while indexing, and don't store those associations in the generated index file either. so you're trading meta names for lower memory usage and a smaller index file. bestie, you're giving up meta names, no cap.-cf,--config-file=fโ the name of the configuration file, f, to use. (default isswish++.confin the current directory.) here's the tea: a config file is not required โ if you don't specify one and the default doesn't exist, none gets used. BUT if you do specify one and it doesn't exist, that's an error. ๐-Cc,--no-class=cโ for HTML or XHTML files only: a class name, c, of an HTML or XHTML element whose text should not be indexed. you can stack multiple-Cor--no-classoptions.-em:p[,p...],--pattern=m:p[,p...]โ a module name, m, and a filename pattern (or a set of patterns separated by commas), p, of files to index. case doesn't matter for the module name, but it totally matters for the patterns. multiple-eor--patternoptions can be specified.-Ep[,p...],--no-pattern=p[,p...]โ a filename pattern (or a set of patterns separated by commas), p, of files not to index. case is significant here. multiple-Eor--no-patternoptions can be specified.-fn,--word-files=nโ the maximum number of files, n, that a word may occur in before it gets discarded for being too frequent. (default is infinity, so like, no limit by default โจ)-Fn,--files-reserve=nโ reserve space for this number of files, n, to start with. more space gets allocated as needed, but with a slight performance penalty. (default is1000.)-gn,--files-grow=nโ grow the space for the reserved number of files, n, when incrementally indexing. the number can be either an absolute number of files or a percentage (when followed by a percent sign%). just like with the-Foption, more space gets allocated as necessary, but with a slight performance penalty. (default is100.)-H,--dump-htmlโ dumps the built-in set of recognized HTML and XHTML elements to standard output and exits. show me the lore ๐-if,--index-file=fโ the name of the generated index file, f (for new indexes; default isswish++.indexin the current directory), or the old index file when you're doing incremental indexing.-I,--incrementalโ incrementally add the indexed files and words to an existing index++. the existing index++ is not touched; instead a new index is created that has the same pathname as the existing index++ with.newappended. no takebacks on the original ๐ซถ-l,--follow-linksโ follow symbolic links during indexing. (default is not to follow them.) heads up: this option is not available under Microsoft Windows since it doesn't support symbolic links.-mm[=n],--meta=m[=n]โ the value of a meta name, m, for which words are to be associated when indexed. case is irrelevant. multiple-mor--metaoptions can be specified.
a meta name can be reassigned when it's followed by a new name, n โ meaning that the name n and not m is stored in the generated index file, so queries would use the new name rather than the original.
by default, words associated with all meta names are indexed. specifying at least one meta name via this option changes that: now only the words associated with a member of the set of meta names explicitly specified via one or more -m or --meta options get indexed. choose your fighters carefully ๐
-Mm,--no-meta=mโ the value of a meta name, m, for which words are not to be indexed. case is irrelevant. multiple-Mor--no-metaoptions can be specified.-pn,--word-percent=nโ the maximum percentage, n, of files a word may occur in before it gets discarded for being too frequent. (default is100.) and if you want to keep all words regardless? specify101. periodt.-P,--no-pos-dataโ do not store word positions in memory during indexing, nor in the generated index file needed to donearsearches later during searching. so you're sacrificingnearsearching for lower memory usage and a smaller index file (approximately50%). bestie, that's a big trade ๐ญ-r,--no-recurseโ do not recursively index the files in subdirectories. meaning: when a directory is encountered, all the files in that directory get indexed (modulo the filename patterns specified via either the-e,--pattern,-E, or--no-patternoptions or theIncludeFileorExcludeFilevariables), but subdirectories encountered are ignored, and therefore the files inside them are not indexed. this option is most useful when you're specifying the directories and files to index via standard input. (default is to index the files in subdirectories recursively.)-sf,--stop-file=fโ the name of a file, f, containing the set of stop-words to use instead of the built-in set. whitespace, including blank lines, and characters starting with#and continuing to the end of the line (comments) are ignored.-S,--dump-stopโ dumps the built-in set of stop-words to standard output and exits. stop-word lore drop ๐-tn,--title-lines=nโ the maximum number of lines, n, into a file to look at for a file's title. (default is12.) bigger numbers slow indexing down.-Td,--temp-dir=dโ the path of the directory, d, to use for temporary files. the directory must exist. (default is/tmpfor Unix or/tempfor Windows.)
โ ๏ธ if your OS mounts swap space on/tmp, then as indexing progresses and more files get created in/tmp, you'll have less swap space, indexing will get slower, and you may run out of memory. if that's your situation, specify a directory on a real filesystem, i.e., one on a physical disk. no cap, save yourself.
-vn,--verbosity=nโ the verbosity level, n, for printing additional information to standard output during indexing. the verbosity levels,0โ4, are:0โ no output is generated except for errors. (this is the default.)1โ only run statistics (elapsed time, number of files, word count) are printed.2โ directories are printed as indexing progresses.3โ directories and files are printed with a word-count for each file.4โ same as3but also prints all files that are not indexed and why. the full tea โ
-V,--versionโ prints the version number of SWISH++ to standard output and exits. slay.-Wn,--word-threshold=nโ the word count past which partial indices are generated and merged, since all the words are too big to fit into memory at the same time. if you index++ and your machine begins to swap like mad, lower this value. only the super-user can specify a value larger than the compiled-in default. ๐
๐ CONFIGURATION FILE ๐
the following variables can be set in a configuration file. variables and command-line options can be mixed, and the command-line options take priority. (yes, the CLI wins. main character energy.)
AssociateMetaโ same as-Aor--no-assoc-metaExcludeClassโ same as-Cor--no-classExcludeFileโ same as-Eor--no-patternExcludeMetaโ same as-Mor--no-metaFilesGrowโ same as-gor--files-growFilesReserveโ same as-For--files-reserveFilterAttachmentโ (see FILTERS inswish++.conf(5).)FilterFileโ (see FILTERS inswish++.conf(5).)FollowLinksโ same as-lor--follow-linksIncludeFileโ same as-eor--patternIncludeMetaโ same as-mor--metaIncrementalโ same as-Ior--incrementalIndexFileโ same as-ior--index-fileRecurseSubdirsโ same as-ror--no-recurseStopWordFileโ same as-sor--stop-fileStoreWordPositionsโ same as-Por--no-pos-dataTempDirectoryโ same as-Tor--temp-dirTitleLinesโ same as-tor--title-linesVerbosityโ same as-vor--verbosityWordFilesMaxโ same as-for--word-filesWordPercentMaxโ same as-por--word-percentWordsNearโ same as-nor--nearWordThresholdโ same as-Wor--word-threshold
โจ EXAMPLES โจ
These are the recipes bestie, and they're all assuming you already cd'd into your web server's document root before you start indexing. Don't skip that part, it's giving prerequisite energy ๐
๐ฅ๏ธ Unix Command-Lines
Wanna index every HTML and text file on a web server? Say less:
index++ -v3 -e 'html:*.*htm*' -e 'text:*.txt' .
Wanna index everything that's not hiding inside a directory called CVS? Pipe it in, queen:
find . -name CVS -prune -o -type f -a -print | index++ -e 'html:*.*htm*' -
๐ช Windows Command-Lines
Okay so when you're in the Windows command interpreter, single quotes around filename patterns simply do not work. You must use double quotes:
index++ -v3 -e "html:*.*htm*" -e "text:*.txt" .
And just so we're clear: this is a Windows problem, not an SWISH++ problem. Not SWISH++'s fault, she's innocent ๐ (Also, double quotes work fine under Unix too, so if you want one blessed syntax forever, there's your answer.)
๐ Using CLASS Attributes to Index HTML Selectively
Sometimes an HTML or XHTML doc has chunks you truly do not want indexed. Classic example: every page on a site has a nav menu like this:
<SELECT NAME="menu">
<OPTION>Home
<OPTION>Automotive
<OPTION>Clothing
<OPTION>Hardware
</SELECT>
โฆor a shared header and footer. Ordinarily those words get indexed on every single page, which means they'd end up so frequent they'd just get thrown out anyway. Waste of energy tbh.
But here's the slay: you can name one or more class names using either the -C / --no-class option or the ExcludeClass variable, and then any HTML or XHTML element belonging to one of those classes will have its text left alone โ nothing gets indexed up to the tag that ends it.
So give a class the name no_index, and that menu above becomes:
<SELECT NAME="menu" CLASS="no_index">
and boom โ everything up to the </SELECT> tag is not indexed. Periodt.
Now for an HTML element that has an optional end tag (hi, <P> element ๐), the text up to whatever tag ends it is not indexed โ and that "whatever tag" can be the element's own end tag or a tag from some other element that implicitly ends it. Watch:
<P CLASS="no_index">
This was the poem that Alice read:
<BLOCKQUOTE>
<B>Jabberwocky</B><BR>
`Twas brillig, and the slithy toves<BR>
Did gyre and gimble in the wabe;<BR>
All mimsy were the borogoves,<BR>
And the mome raths outgrabe.
</BLOCKQUOTE>
The <BLOCKQUOTE> tag implicitly ends the <P> element โ as do all block-level elements, just so you know โ so the only text above that is not indexed is: ``This was the poem that Alice read.''
And for an HTML or XHTML element that has no end tag at all? Only the text inside the start tag itself stays unindexed. Behold:
<IMG SRC="home.gif" ALT="Home" CLASS="no_index">
The word ``Home'' will not be indexed โ even though it absolutely would have been if that CLASS attribute weren't sitting there. Iconic behavior, honestly.
๐ง Filters
(Go look at Filters under EXAMPLES in swish++.conf(5) โ they live over there.)
๐ EXIT STATUS ๐
It bails out with one of these bad boys:
0โ Success. We ate, we left no crumbs ๐1โ Error in configuration file.2โ Error in command-line options.10โ Unable to open temporary file.11โ Unable to write index file.12โ Unable to write temporary file.13โ Root-only operation attempted.30โ Unable to read stop-word file.40โ Unable to read index file.127โ Internal error. ๐ญ
โ ๏ธ CAVEATS โ ๏ธ
- Generated index files are machine-dependent โ that means the size of data types and byte order matter. No porting these around like they're nothing, bestie.
- The word-determination heuristics are heavily geared for English. Using SWISH++ as-is to index and
search++files in non-English languages is not recommended. Fr fr, don't. - Unless it says otherwise somewhere up above, the character encoding always used is ISO 8859-1 (Latin 1). Any character encodings specified inside HTML or XHTML files are ignored. Ignored!! ๐
- An e-mail message can have both an encoding and a non-ASCII or non-ISO-8859-1 charset at the same time โ like
base64-encoded UTF-8. (In practice this specific case should never happen, since UTF-7 should be used instead; but you get the idea.)
But handling an encoding and such a charset simultaneously is problematic, which is why an e-mail message or attachment can have either an encoding or a non-ASCII or a non-ISO-8859-1 character set โ but not both. If it somehow does, the encoding takes precedence. Encoding wins, always, no negotiation.
๐ FILES ๐
swish++.confโ default configuration file nameswish++.indexโ default index file name
๐ ENVIRONMENT ๐
TMPDIRโ If it's set, this is the default path of the directory used for temporary files. The directory must exist. This gets superseded by either the-T/--temp-diroption or theTempDirectoryvariable.
๐ SEE ALSO ๐
And now the reading list, because this page is nothing if not well-cited ๐
- Tim Berners-Lee. ``The text/enriched MIME Content-type,'' Request for Comments 1563, Network Working Group of the Internet Engineering Task Force, January 1994.
- David H. Crocker. ``Standard for the Format of ARPA Internet Text Messages,'' Request for Comments 822, Department of Electrical Engineering, University of Delaware, August 1982.
- Frank Dawson and Tim Howes. ``vCard MIME Directory Profile,'' Request for Comments 2426, Network Working Group of the Internet Engineering Task Force, September 1998.
- Ned Freed and Nathaniel S. Borenstein. ``Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies,'' Request for Comments 2045, RFC 822 Extensions Working Group of the Internet Engineering Task Force, November 1996.
- David Goldsmith and Mark Davis. ``UTF-7, a mail-safe transformation format of Unicode,'' Request for Comments 2152, Network Working Group of the Internet Engineering Task Force, May 1997.
- International Standards Organization. ISO 8859-1: Information Processing -- 8-bit single-byte coded graphic character sets -- Part 1: Latin alphabet No. 1, 1987.
- --. ISO 8879: Information Processing -- Text and Office Systems -- Standard Generalized Markup Language (SGML), 1986.
- --. ISO/IEC 9945-2: Information Technology -- Portable Operating System Interface (POSIX) -- Part 2: Shell and Utilities, 1993.
- Leslie Lamport. LaTeX: A Document Preparation System, 2nd ed., Addison-Wesley, Reading, MA, 1994.
- Martin Nilsson. ID3 tag version 2, March 1998.
- --. ID3 tag version 2.3.0, February 1999.
- --. ID3 tag version 2.4.0 - Main Structure, November 2002.
- --. ID3 tag version 2.4.0 - Native Frames, November 2002.
- Steven Pemberton, et al. XHTML 1.0: The Extensible HyperText Markup Language, World Wide Web Consortium, January 2000.
- Dave Raggett, Arnaud Le Hors, and Ian Jacobs. ``On SGML and HTML: SGML constructs used in HTML: Entities,'' HTML 4.0 Specification, ยง3.2.3, World Wide Web Consortium, April 1998.
- --. ``The global structure of an HTML document: The document head: The title attribute,'' HTML 4.0 Specification, ยง7.4.3, World Wide Web Consortium, April 1998.
- --. ``The global structure of an HTML document: The document head: Meta data,'' HTML 4.0 Specification, ยง7.4.4, World Wide Web Consortium, April 1998.
- --. ``The global structure of an HTML document: The document body: Element identifiers: the id and class attributes,'' HTML 4.0 Specification, ยง7.5.2, World Wide Web Consortium, April 1998.
- --. ``Tables: Elements for constructing tables: The TABLE element,'' HTML 4.0 Specification, ยง11.2.1, World Wide Web Consortium, April 1998.
- --. ``Objects, Images, and Applets: Generic inclusion: the OBJECT element,'' HTML 4.0 Specification, ยง13.3, World Wide Web Consortium, April 1998.
- --. ``Objects, Images, and Applets: How to specify alternate text,'' HTML 4.0 Specification, ยง13.8, World Wide Web Consortium, April 1998.
- --. ``Index of Elements,'' HTML 4.0 Specification, World Wide Web Consortium, April 1998.
- Marcin Sawicki, et al. Ruby Annotation, World Wide Web Consortium, April 2001.
- The Unicode Consortium. ``Encoding Forms,'' The Unicode Standard 3.0, ยง2.3, Addison-Wesley, 2000.
- Francois Yergeau. ``UTF-8, a transformation format of ISO 10646,'' Request for Comments 2279, Network Working Group of the Internet Engineering Task Force, January 1998.
๐ AUTHOR ๐
Paul J. Lucas <pauljlucas@mac.com> โ the actual main character, no notes ๐ซถ
Okay and that's the whole page!! You made it to the end, I'm so proud of you ๐ฅน This little masterpiece is from SWISH++, dated March 25, 2004 โ and that's index++(1), signed, sealed, delivered. Go index something, bestie ๐
โจ
๐ฆ where this page comes from
- debian bookworm: swish++ 6.1.5-6
- debian trixie: swish++ 6.1.5-6
- ubuntu jammy: swish++ 6.1.5-6
- ubuntu noble: swish++ 6.1.5-6build1
- ubuntu questing: swish++ 6.1.5-6build1
- ubuntu resolute: swish++ 6.1.5-6build2
- ubuntu stonking: swish++ 6.1.5-6build2
- Copyright (C) 2003 Paul J. Lucas
- ๐ girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms