STXXL_TOOL(1)

💖 STXXL_TOOL 💖

okay bestie, gather round, because this is the STXXL scratch file administration and performance testing tool and it is a lot. it lives in the General Commands Manual section (that's section 1, iykyk), and honestly? it's giving swiss army knife energy. 💅

---

✨ SYNOPSIS ✨

stxxl_tool <subtool> ...

---

🌸 DESCRIPTION 🌸

so stxxl_tool is basically the umbrella program of the family — one entry point that houses a whole lil squad of sub-tools for STXXL, handling stuff like (e.g.) scratch file manipulation and benchmarking. each sub-tool has its own whole personality, and we're gonna go through every single one below, no skips, no cap. 🫶

oh and size arguments are super chill about formatting — you can hand them a string like "343KB" or "44 GiB" and they'll just get it. iconic behaviour honestly.

🎀 Subtools 🎀

🌈 info

prints out the tea on the build system and which optional modules got compiled into STXXL. like a little birth certificate for your build. 💖

🦋 create_files <filesize> <filename ...>

pre-creates big files so that file system allocation time doesn't sneak into your measurements and ruin the vibe. we're keeping the numbers clean, periodt.

🔥 benchmark_disks [options] <size> [r|w] [alloc]

benchmarks the disks that are configured by the standard .stxxl disk configuration files mechanism. blocks of 8 MiB get written and/or read in sequence using the block manager. the batch size describes how many blocks get written/read in one batch. those are taken from block_manager using the specified allocation strategy. and if size == 0, then the writing/reading operations just keep going until an error occurs (chaotic, but okay). 😭

Parameters:

• size — how much data to write/read from the disks (e.g. 10GiB)
• r|w — only read or only write blocks (default: both write and read)
• alloc — block allocation strategy: RC, SR, FR, striping. (default: RC)

Options:

• -b, --batch — number of blocks written/read in one batch (default: D * 8MiB)

🍓 benchmark_files [options] <length> <filename ...>

opens a file using one of STXXL's file abstractions and runs write/read/verify tests on it. block sizes and batch size are adjustable from the command line, because we love options. if length == 0, then the operation keeps going till the end of space (and yes, bestie, please just ignore the write error). memory consumption goes like block_size * batch_size * num_files. 🧚

Parameters:

• length — length to write in the file.
• filename — file path to run the benchmark on.

Options:

• -o, --offset — starting offset to write in the file.
• --no-direct — open files without O_DIRECT
• --sync — open files with O_SYNC|O_DSYNC|O_RSYNC
• --resize — resize the file size after opening, needed e.g. for creating mmap files
• --block_size — block size for operations (default 8 MiB)
• --batch_size — increase (default 1) to submit several I/Os at once and report average rate
• -f, --file-type — method to open the file (syscall|mmap|wincall|boostfd|...), default: syscall
• -p, --operations — [w]rite pattern, [r]ead without verification, read and [v]erify pattern (default: 'wv')
• --pattern — 32-bit pattern to write (default: block index)

and yeah, this one is literally built to benchmark different file access methods, e.g. syscall or mmap_files. slay. ✨

👑 benchmark_sort [options] <size>

runs benchmark tests of the different sorting methods in STXXL. size is the amount of data to sort, in GiB.

Valid options are:

• -M, --ram — amount of ram to use when sorting, default 256 MiB

💀 benchmark_disks_random <span> [block_size] [size] [i|r|w] [alloc]

benchmarks _random_ block access on the disks configured by the standard .stxxl disk configuration files mechanism. available block sizes are powers of two from 4 KiB to 128 MiB. there's a set of three operations that can be performed: sequential initialization, random reading and random writing. choose your fighter. 💅

Parameters:

• span — span of external memory to write/read to (e.g. 10GiB).
• block_size — size of blocks to randomly write/read (default: 8MiB).
• size — amount of data to operate on (e.g. 2GiB), default: whole span.
• i|r|w — operations: [i]nitialize, [r]ead, and/or [w]rite (default: all).
• alloc — block allocation strategy: RC, SR, FR, striping (default: RC).

🙈 benchmark_pqueue [options] [size]

benchmarks the priority queue implementation using a sequence of operations. the PQ contains pairs of 32- or 64-bit integers, or a 24 byte struct. the operation sequence is either a simple fill/delete cycle or fill/intermixed inserts/deletes. and here's the catch bestie — because the memory parameters of the PQ have to be set at compile-time, the benchmark only offers three PQ sizes: for 256 MiB, 1 GiB and 8 GiB of RAM, with the maximum number of items set accordingly. 💖

Parameters:

• size — amount of data to insert (e.g. 1GiB)

Options:

• -t, --type — value type of the tested priority queue:
  • 1 = pair of uint32,
  • 2 = pair of uint64 (default),
  • 3 = 24 byte struct
  • 0 = all of the above
• -p, --pq — priority queue configuration to test:
  • 1 = small (256 MiB RAM, 4 GiB elements)
  • 2 = medium (1 GiB RAM, 16 GiB elements) (default)
  • 3 = big (8 GiB RAM, 64 GiB elements)
  • 0 = all of the above
• -o, --opseq — operation sequence to perform:
  • 1 = insert all, delete all (default)
  • 2 = insert all, intermixed insert/delete
  • 0 = all of the above

⭐ mlock <size>

allocates some memory and mlock()s it, to consume physical memory. fun little memory hog. but heads up:

it needs to run as root to block more than 64 KiB in default settings. so no, your regular user can't go feral with this one. 😭

🌸 mallinfo

shows mallinfo statistics. that's it, that's the whole subtool. short and sweet. 🎀

---

💜 AUTHOR 💜

stxxl is written by many authors, and you can find all of them in the AUTHORS section of the documentation package, or on the homepage. group project energy fr fr. 🫶

---

🌈 HOMEPAGE 🌈

libstxxl Project Home : http://stxxl.sourceforge.net

this manual page was written by D Haley <mycae gmx com>, for the Debian project (and may be used by others). bestie really said "I'll document it myself." iconic. 👑

---

and that's the whole thing, bestie — December 16, 2013, general commands manual, STXXL_TOOL, and you now know every single subtool, flag and parameter. go forth and benchmark responsibly. 💅✨🎀