MU-INDEX(1)

πŸ“¦
version 1.14.2 Β· 11 versions
, in ubuntu stonking

πŸ’– NAME πŸ’–

okay so bestie, mu-index β€” it indexes your e-mail messages that live in Maildirs. that's the whole job. queen behavior ✨

πŸ¦‹ SYNOPSIS πŸ¦‹

mu [COMMON-OPTIONS] index

🌸 DESCRIPTION 🌸

so mu index is the mu command that goes around scanning all the messages in your Maildir directories and storing what it finds in a Xapian database. once it's in there, you can go query it with mu-find(1). that's the pipeline, no cap πŸ’…

before you can even run mu index, you have to initialize mu and its database with mu-init(1). like, you can't skip that step, bestie, it's giving "read the instructions first."

mu index understands Maildirs as defined by Daniel Bernstein for qmail(7). and more than that β€” it also gets recursive Maildirs (that's Maildirs nestled inside Maildirs, cute right?), Maildir++, and it supports VFAT-based Maildirs which use "!" or ";" as the separators instead of the standard ":". versatile icon ✨

e-mail messages only get considered for indexing if they live in a directory named either cur or new. the special tmp directory (special as per the Maildir-specification) is ignored, and so are some cache directories for some other mail-clients. everything else? scanned recursively. periodt.

symbolic links get followed, and the directories can be spread over multiple filesystems β€” however, quick note: moving files around is much faster when they all live on a single file-system. and please, please be careful to avoid self-referential symlinks! that's the πŸ’€ part.

if there's a file called .noindex in a directory, then the contents of that directory and all of its subdirectories will be ignored. super useful for excluding certain directories from the indexing process β€” for example, directories full of spam-messages. bye πŸ‘‹

if there's a file called .noupdate in a directory, the contents of that directory and all of its subdirectories will be ignored too. this one's for speeding things up when you've got some maildirs that literally never change.

important tho: .noupdate does not affect already-indexed messages β€” you can still search for them, girl, they're fine. and .noupdate is ignored when you start indexing with an empty database (like, directly after mu init).

there's also the option --lazy-check which can greatly speed up indexing; details below, keep reading 🫢

a first run of mu index may take a few minutes if you have a lot of mail (tens or hundreds of thousands of messages β€” yeah, real ones). fortunately, that big full scan only has to happen rarely, so don't panic. after that, it suffices to just index the changes, which goes way faster. see PERFORMANCE (i,ii,iii) below for more info.

the optional cleanup-phase of the indexing-process is the removal of messages from the database for which there's no longer a corresponding file in the Maildir. if you don't want that, use -n, --nocleanup.

when mu index catches one of the signals SIGINT, SIGHUP or SIGTERM (like when you press Ctrl+C during the indexing process), it tries to shut down gracefully: save and commit data, close the database, etc. but if it receives another signal (e.g., when you press Ctrl+C once more), mu index will terminate immediately. so like, one is a polite request, two is a vibe check failure 😭

πŸ‘‘ INDEX OPTIONS πŸ‘‘

  • --lazy-check β€” in lazy-check mode, mu does not consider messages for which the time-stamp (ctime) of the directory they reside in hasn't changed since the previous time this directory was checked. it's much faster than the non-lazy check, but it won't update messages that have changed (as opposed to being added or removed), since merely editing a message doesn't update the directory time-stamp. naturally, you can run mu-index occasionally without --lazy-check to pick up those messages. also, in lazy-check mode, files which have a ctime smaller than the time the previous indexing operation was completed get ignored β€” which helps for the use-case where new messages can appear in big maildirs. so smart πŸ’…
  • --nocleanup β€” disable the database cleanup that mu does by default after indexing.
  • --reindex β€” perform a complete reindexing of all the messages in the maildir. everything. from scratch. πŸ”₯
  • --muhome β€” use a non-default directory to store and read the database, write the logs, etc. by default, mu uses the XDG Base Directory Specification (e.g. on GNU/Linux this defaults to ~/.cache/mu and ~/.config/mu). earlier versions of mu defaulted to ~/.mu, which now requires --muhome=~/.mu.

the environment variable MUHOME can be used as an alternative to --muhome. the latter has precedence, just so you know πŸŽ€

✨ COMMON OPTIONS ✨

  • -d, --debug β€” makes mu generate extra debug information, useful for debugging the program itself. debug information goes to the standard logging location; see mu(1).
  • -q, --quiet β€” causes mu not to output informational messages and progress information to standard output, but only to the log file. error messages will still be sent to standard error. heads up bestie: mu index is much faster with --quiet, so it's recommended you use this option when using mu from scripts etc. like, actually use it.
  • --log-stderr β€” causes mu to not output log messages to standard error, in addition to sending them to the standard logging location.
  • --nocolor β€” do not use ANSI colors. the environment variable NO_COLOR can be used as an alternative to --nocolor.
  • -V, --version β€” prints mu version and copyright information.
  • -h, --help β€” lists the various command line options.

πŸ”’ ENCRYPTION πŸ”’

mu index does not decrypt messages. only the metadata (such as headers) of encrypted messages makes it to the database. mu view and mu4e can decrypt messages, but those work with the message directly and that information is not added to the database. so no, your secrets stay secret πŸ™ˆ

πŸš€ PERFORMANCE πŸš€

⭐ indexing in ancient times (2009?)

as a non-scientific benchmark, a simple test on the author's machine (a Thinkpad X61s laptop using Linux 2.6.35 and an ext3 file system) with no existing database, and a maildir with 27273 messages:

$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ time mu index --quiet
66,65s user 6,05s system 27% cpu 4:24,20 total

(about 103 messages per second β€” yeah. we've come a long way πŸ₯²)

a second run, which is the more typical use case when there's a database already, goes much faster:

$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ time mu index --quiet
0,48s user 0,76s system 10% cpu 11,796 total

(more than 56818 messages per second ✨)

note that each test flushes the caches first; a more common use case might be to run mu index when new mail has arrived; the cache may stay quite warm in that case:

$ time mu index --quiet
0,33s user 0,40s system 80% cpu 0,905 total

which is more than 30000 messages per second. slay.

⭐ indexing in 2012

as per June 2012, we did the same non-scientific benchmark, this time with an Intel i5-2500 CPU @ 3.30GHz, an ext4 file system and a maildir with 22589 messages. we start without an existing database.

$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ time mu index --quiet
27,79s user 2,17s system 48% cpu 1:01,47 total

(about 813 messages per second)

a second run, which is the more typical use case when there's a database already, goes much faster:

$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ time mu index --quiet
0,13s user 0,30s system 19% cpu 2,162 total

(more than 173000 messages per second πŸ”₯)

⭐ indexing in 2016

as per July 2016, we did the same non-scientific benchmark, again with the Intel i5-2500 CPU @ 3.30GHz, an ext4 file system. this time, the maildir contains 72525 messages.

$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ time mu index --quiet
40,34s user 2,56s system 64% cpu 1:06,17 total

(about 1099 messages per second).

⭐ indexing in 2022

a few years later and it's June 2022. there's a lot more happening during indexing now, BUT indexing became multi-threaded and machines are faster; e.g. this is with an AMD Ryzen Threadripper 1950X (16 cores) @ 3.399GHz.

the instructions are a little different since we have a proper repeatable benchmark now. after building,

$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
% THREAD_NUM=4 build/lib/tests/bench-indexer -m perf
# random seed: R02Sf5c50e4851ec51adaf301e0e054bd52b
1..1
# Start of bench tests
# Start of indexer tests
indexed 5000 messages in 20 maildirs in 3763ms; 752 ΞΌs/message; 1328 messages/s (4 thread(s))
ok 1 /bench/indexer/4-cores
# End of indexer tests
# End of bench tests

things are again a little faster, even though the index does a lot more now (text-normalization, and pre-generating message-sexps). a faster machine helps, too! πŸ’…

⭐ recent releases

indexing the same 93000-message mail corpus with the last few releases:

  • release 1.4 β€” time (sec): 160s
  • release 1.6 β€” time (sec): 178s
  • release 1.8 β€” time (sec): 97s
  • release 1.10 β€” time (sec): 120s β€” notes: adds html indexing, sexp-caching
  • release 1.11 (master) β€” time (sec): 96s β€” notes: adds language-guessing, batch-size=50000

quite some variation!

over time new features / refactoring can change the timings quite a bit. at least for now, the latest code is both the fastest and the most feature-rich. main character behavior honestly πŸ‘‘

πŸ’€ EXIT CODE πŸ’€

this command returns 0 upon successful completion, or a non-zero exit code otherwise.

  1. 0. success ✨
  2. 2. no matches found. Try a different query
  3. 11. database schema mismatch. You need to re-initialize mu, see mu-init(1)
  4. 19. failed to acquire lock. Some other program has exclusive access to the mu database
  5. 99. caught an exception

πŸ™ˆ REPORTING BUGS πŸ™ˆ

please report bugs at https://github.com/djcb/mu/issues.

🌈 AUTHOR 🌈

Dirk-Jan C. Binnema <djcb@djcbsoftware.nl>

this manpage is part of mu 1.14.2.

Copyright Β© 2008-2026 Dirk-Jan C. Binnema. License GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

🫢 SEE ALSO 🫢

maildir(5), mu(1), mu-init(1), mu-find(1), mu-cfind(1)

and that's the whole page, bestie β€” MU INDEX(1), from the General Commands Manual, part of mu 1.14.2. go index something. ily πŸ’–βœ¨πŸ¦‹

πŸ“¦ where this page comes from