extract++(1)

💖 extract++(1) — the SWISH++ text extractor, but make it a vibe 💖

okay bestie, buckle up, because this is extract++ from the SWISH++ family, and it lives in the General Commands Manual section. it's giving "I pull text out of files that are absolutely not text files." 💅

---

✨ NAME ✨

extract++ — the SWISH++ text extractor. yeah. that's her. 🎀

---

🌸 SYNOPSIS 🌸

extract++ [ options ] directory...  file...

---

🦋 DESCRIPTION 🦋

so extract++ is the SWISH++ text extractor, a utility that extracts whatever text there is from a (mostly) binary file — think of it like strings(1) but with a glow-up — and it does this prior to indexing. and the best part? Original files are untouched. we do not touch. we only look. 👑

Text gets extracted from the files you specify and from the files inside the directories you specify. and by default, text from files in subdirectories of specified directories gets extracted too — UNLESS you pass the -r, --no-recurse, -f, or --filter option, or you set the RecurseSubdirs or ExtractFilter variable. then the subdirectory party is cancelled. 🙈

Ordinarily, text is extracted from files either only if their filename matches one of the patterns in the set you specified with the -e or --pattern option or the IncludeFile variable (unless standard input is being used — more on that in literally the next paragraph, bestie, don't worry) or if the file is not among the set you specified with the -E or --no-pattern option or the ExcludeFile variable.

If there is a single filename of -, then the list of directories and files to extract is instead taken from standard input, one per line. and in that case, filename patterns of files to extract don't even need to be specified explicitly: all files, regardless of whether they match a pattern — unless they're among the set not to extract specified with the -E or --no-pattern option or the ExcludeFile variable — are extracted. i.e., extract++ assumes you know what you're doing when you specify filenames this way. main character energy, honestly. 💅

Ordinarily, the text extracted from a file is written to another file in the same directory, having the same filename but with the ``.txt'' extension appended by default. so like, foo.doc becomes foo.doc.txt after extraction. (see also the -x or --extension option or the ExtractExtension variable.) HOWEVER — and this is important — extraction is not performed if the extracted text file already exists. she said no double work. 😭

If either the -f or --filter option or the ExtractFilter variable is given, then only a single file specified on the command line is extracted to standard output and we're done. in this case, filename patterns are not used and the existence of an extracted text file is irrelevant. no patterns, no problems. ✨

🎀 Filters 🎀

Via the FilterFile configuration file variable, files having particular patterns can be filtered prior to extraction. (see the examples in swish++.conf(5).) 🧚

🌈 Character Mapping and Word Determination 🌈

extract++ performs the same character mapping, character entity conversions, and word determination heuristics used by index++(1) — but she also additionally does these three things:

1. Considers all PostScript Level 2 operators that are not also English words to be stop words. such words in a file usually indicate an encapsulated PostScript (EPS) file, and such should not be indexed. not on my watch. 💀
2. Looks specifically for encapsulated PostScript (EPS) data between everything between one of %%BeginSetup, %%BoundingBox, %%Creator, %%EndComments, or %%Title and %%Trailer, and discards it. bye. 🫶
3. Discards strings of ASCII hex data Word_Hex_Min_Size characters or longer, e.g., 7F454C46. (Default is 5.)

💜 Motivation 💜

extract++ was developed to be able to index non-text files in proprietary formats such as Microsoft Office documents. there are a couple of reasons why the functionality of extract++ isn't simply built into index++(1):

1. Users who do not need to index such documents shouldn't have to pay the performance penalty for doing the extra checks for PostScript and hex data. fairness!! ⭐
2. While index++(1) can uncompress files on the fly using filters also, uncompressing them every time indexing is performed is excessive. Text extraction, on the other hand, is done only once per file; if the file is updated, the text-extracted version should be deleted and recreated. that's the whole business model, bestie. 🍓

---

✨ OPTIONS ✨

okay so, house rules first, because this parser has opinions:

Options begin with either a - for short options or a --'' for long options. Either a `-` or --'' by itself explicitly ends the options; however, the difference is that - is returned as the first non-option whereas ``--'' is skipped entirely. Long option names may be abbreviated so long as the abbreviation is unambiguous.

For a short option that takes an argument, the argument is either taken to be the remaining characters of the same option, if any, or, if not, is taken from the next option unless said option begins with a -.

Short options that take no arguments can be grouped (but the last option in the group can take an argument), e.g., -lrv4 is equivalent to -l -r -v4.

For a long option that takes an argument, the argument is either taken to be the characters after a =, if any, or, if not, is taken from the next option unless said option begins with a -.

---

• -?, --help — print the usage (``help'') message and exit. 🫶
• -cc, --config-file=c — the name of the configuration file, c, to use. (Default is swish++.conf in the current directory.) A configuration file is not required: if none is specified and the default does not exist, none is used; however, if one is specified and it does not exist, then this is an error. no ghost configs allowed. 👻
• -ep[,p...], --pattern=p[,p...] — a filename pattern (or set of patterns separated by commas), p, of files to extract text from. Case is significant. Multiple -e or --pattern options may be specified. stack 'em. 🔥
• -Ep[,p...], --no-pattern=p[,p...] — a filename pattern or patterns, p, of files not to extract text from. Case is significant. Multiple -E or --no-pattern options may be specified. 💅
• -f, --filter — extract a single file to standard output and exit. one file, one shot, we're out. ✨
• -l, --follow-links — follow symbolic links during extraction. The default is not to follow them. (This option is not available under Microsoft Windows since it doesn't support symbolic links.) sorry windows girlies. 😭
• -r, --no-recurse — do not recursively extract the files in subdirectories, that is: when a directory is encountered, all the files in that directory are extracted (modulo the filename patterns specified via the -e, --pattern, -E, or --no-pattern options or the IncludeFile or ExcludeFile variables) but subdirectories encountered are ignored and therefore the files contained in them are not extracted. (This option is most useful when specifying the directories and files to extract via standard input.) The default is to extract the files in subdirectories recursively. 🦋
• -sf, --stop-file=f — the name of a file, f, containing the set stop-words to use instead of the built-in set. Whitespace, including blank lines, and characters starting with # and continuing to the end of the line (comments) are ignored. 🎀
• -S, --dump-stop — dump the built-in set of stop-words to standard output and exit. show me the stop-words, bestie. 👑
• -vc, --verbosity=v — the verbosity level, v, for printing additional information to standard output during indexing. The verbosity levels, 0-4, are:
  • 0 — No output is generated (except for errors).
  • 1 — Only run statistics (elapsed time, number of files, word count) are printed.
  • 2 — Directories are printed as extraction progresses.
  • 3 — Directories and files are printed with a word-count for each file.
  • 4 — Same as 3 but also prints all files that are not extracted and why. the full tea. ☕
• -V, --version — print the version number of SWISH++ and exit. 💖
• -xe, --extension=e — the extension to append to filenames during extraction. (It can be specified with or without the dot; default is txt.) 🍓

---

🌸 CONFIGURATION FILE 🌸

The following variables can be set in a configuration file. Variables and command-line options can be mixed. live your truth.

ExcludeFile       Same as -E or --no-pattern
ExtractExtension  Same as -x or --extension
ExtractFilter     Same as -f or --filter
FilterAttachment  (See FILTERS in swish++.conf(5).)
FilterFile        (See FILTERS in swish++.conf(5).)
FollowLinks       Same as -l or --follow-links
IncludeFile       Same as -e or --pattern
RecurseSubdirs    Same as -r or --no-recurse
StopWordFile      Same as -s or --stop-file
Verbosity         Same as -v or --verbosity

---

🧚 EXAMPLES 🧚

🌈 Extraction 🌈

To extract text from all Microsoft Office files on a web server:

cd /home/www/htdocs
extract++ -v3 -e '*.doc' -e '*.ppt' -e '*.xls' .

look at her go. 💅

🎀 Filters 🎀

(See the examples in swish++.conf(5).) 🫶

---

⭐ EXIT STATUS ⭐

Exits with one of the values given below:

0    Success.
1    Error in configuration file.
2    Error in command-line options.
20   File to extract does not exist.
30   Unable to read stop-word file.

---

💀 CAVEATS 💀

1. Text extraction is not perfect, nor can be. she's doing her best. 😭
2. As with index++(1), the word-determination heuristics employed are heavily geared for English. Using SWISH++ as-is to extract files in non-English languages is not recommended. (iykyk.)

---

💜 FILES 💜

swish++.conf      default configuration file name

---

🦋 SEE ALSO 🦋

index++(1), search++(1), strings(1), swish++.conf(5), glob(7)

Adobe Systems Incorporated. PostScript Language Reference Manual, 2nd ed. Addison-Wesley, Reading, MA. pp. 346-359.

International Standards Organization. ``ISO/IEC 9945-2: Information Technology -- Portable Operating System Interface (POSIX) -- Part 2: Shell and Utilities,'' 1993.

---

👑 AUTHOR 👑

Paul J. Lucas <pauljlucas@mac.com>

---

and that's the whole page, bestie!! this was SWISH++, dated November 1, 2002, and honestly? iconic for it. go forth and extract. 💖✨🎀🦋🌸💅🫶