MILLER(1)

MILLER(1) 🎀✨

NAME 💖

Miller — like awk, sed, cut, join, and sort for name-indexed data such as CSV and tabular JSON. Manual name: Miller, bestie 💅

SYNOPSIS 🦋

Okay so the shape of an mlr call is literally this:

Usage: mlr [flags] [verb] [verb-dependent options ...] {zero or more file names}

If no verb is given, cat is used, which is handy for format conversions, e.g.

mlr --c2j < example.csv

If zero file names are provided, standard input is read, e.g.

mlr --csv sort -f shape example.csv

Output of one verb may be chained as input to another using then, e.g.

mlr --csv stats1 -a min,mean,max -f quantity then sort -f color
example.csv

Please see mlr help topics for more information. Please also see https://miller.readthedocs.io 🫶

DESCRIPTION 💅

Miller operates on key-value-pair data while the familiar Unix tools operate on integer-indexed fields: if the natural data structure for the latter is the array, then Miller's natural data structure is the insertion-ordered hash map. This encompasses a variety of data formats, including but not limited to the familiar CSV, TSV, and JSON. (Miller can handle positionally-indexed data as a special case.) This manpage documents mlr 6.22.0. No cap, that's the whole vibe ✨

EXAMPLES 🌈

Here's the little runway walk of examples, bestie:

mlr --icsv --opprint cat example.csv
mlr --icsv --opprint sort -f shape example.csv
mlr --icsv --opprint sort -f shape -nr index example.csv
mlr --icsv --opprint cut -f flag,shape example.csv
mlr --csv filter '$color == "red"' example.csv
mlr --icsv --ojson put '$ratio = $quantity / $rate' example.csv
mlr --icsv --opprint --from example.csv sort -nr index then cut -f shape,quantity

FILE FORMATS 🌸

Miller speaks a whole buffet of formats, and here's the visual tea ☕

• CSV/CSV-lite: comma-separated values with separate header line
• TSV: same but with tabs in places of commas

+---------------------+
| apple,bat,cog       |
| 1,2,3               | Record 1: "apple":"1", "bat":"2", "cog":"3"
| 4,5,6               | Record 2: "apple":"4", "bat":"5", "cog":"6"
+---------------------+

JSON (array of objects):

+---------------------+
| [                   |
| {                   |
|  "apple": 1,        | Record 1: "apple":"1", "bat":"2", "cog":"3"
|  "bat": 2,          |
|  "cog": 3           |
| },                  |
| {                   |
|   "dish": {         | Record 2: "dish.egg":"7",
|     "egg": 7,       | "dish.flint":"8", "garlic":""
|     "flint": 8      |
|   },                |
|   "garlic": ""      |
| }                   |
| ]                   |
+---------------------+

JSON Lines (sequence of one-line objects):

+------------------------------------------------+
| {"apple": 1, "bat": 2, "cog": 3}               |
| {"dish": {"egg": 7, "flint": 8}, "garlic": ""} |
+------------------------------------------------+
  Record 1: "apple":"1", "bat":"2", "cog":"3"
  Record 2: "dish:egg":"7", "dish:flint":"8", "garlic":""

YAML (single document = object or array of objects; multiple documents with ---):

+---------------------+
| apple: 1            |
| bat: 2              | Record 1: "apple":"1", "bat":"2", "cog":"3"
| cog: 3              |
| ---                 |
| dish: 7             | Record 2: "dish":"7", "egg":"8"
| egg: 8              |
+---------------------+

PPRINT: pretty-printed tabular

+---------------------+
| apple bat cog       |
| 1     2   3         | Record 1: "apple:"1", "bat":"2", "cog":"3"
| 4     5   6         | Record 2: "apple":"4", "bat":"5", "cog":"6"
+---------------------+

Markdown tabular:

+-----------------------+
| | apple | bat | cog | |
| | ---   | --- | --- | |
| | 1     | 2   | 3   | | Record 1: "apple:"1", "bat":"2", "cog":"3"
| | 4     | 5   | 6   | | Record 2: "apple":"4", "bat":"5", "cog":"6"
+-----------------------+

XTAB: pretty-printed transposed tabular

+---------------------+
| apple 1             | Record 1: "apple":"1", "bat":"2", "cog":"3"
| bat   2             |
| cog   3             |
|                     |
| dish 7              | Record 2: "dish":"7", "egg":"8"
| egg  8              |
+---------------------+

DKVP: delimited key-value pairs (Miller default format)

+---------------------+
| apple=1,bat=2,cog=3 | Record 1: "apple":"1", "bat":"2", "cog":"3"
| dish=7,egg=8,flint  | Record 2: "dish":"7", "egg":"8", "3":"flint"
+---------------------+

DKVPX: delimited key-value pairs with CSV-style quoting

+----------------------------------+
| apple=1,bat=2,cog=3              | Record 1: "apple":"1", "bat":"2", "cog":"3"
| "x,y"="a,b,c",z=3                | Record 2: "x,y":"a,b,c", "z":"3"
+----------------------------------+

NIDX: implicitly numerically indexed (Unix-toolkit style)

+---------------------+
| the quick brown     | Record 1: "1":"the", "2":"quick", "3":"brown"
| fox jumped          | Record 2: "1":"fox", "2":"jumped"
+---------------------+

DCF: Debian control file format

+------------+
| apple: 1   |
| bat: 2     |
| cog: 3     |
|            |
| dish: 7    |
| egg: 8     |
| 3: flint   |
+------------+

HELP OPTIONS 🫶

Type mlr help {topic} for any of the following:

• Essentials:
  • mlr help topics
  • mlr help basic-examples
  • mlr help file-formats
• Flags:
  • mlr help flags
  • mlr help flag
  • mlr help list-separator-aliases
  • mlr help list-separator-regex-aliases
  • mlr help comments-in-data-flags
  • mlr help compressed-data-flags
  • mlr help csv/tsv-only-flags
  • mlr help dkvp-only-flags
  • mlr help file-format-flags
  • mlr help flatten-unflatten-flags
  • mlr help format-conversion-keystroke-saver-flags
  • mlr help json-only-flags
  • mlr help legacy-flags
  • mlr help markdown-only-flags
  • mlr help miscellaneous-flags
  • mlr help output-colorization-flags
  • mlr help pprint-only-flags
  • mlr help profiling-flags
  • mlr help separator-flags
• Verbs:
  • mlr help list-verbs
  • mlr help usage-verbs
  • mlr help verb
• Functions:
  • mlr help list-functions
  • mlr help list-function-classes
  • mlr help list-functions-in-class
  • mlr help usage-functions
  • mlr help usage-functions-by-class
  • mlr help function
• Keywords:
  • mlr help list-keywords
  • mlr help usage-keywords
  • mlr help keyword
• Other:
  • mlr help auxents
  • mlr help terminals
  • mlr help mlrrc
  • mlr help output-colorization
  • mlr help type-arithmetic-info
  • mlr help type-arithmetic-info-extended
• Shorthands:
  • mlr -g = mlr help flags
  • mlr -l = mlr help list-verbs
  • mlr -L = mlr help usage-verbs
  • mlr -f = mlr help list-functions
  • mlr -F = mlr help usage-functions
  • mlr -k = mlr help list-keywords
  • mlr -K = mlr help usage-keywords

Lastly, mlr help ... will search for your exact text ... using the sources of mlr help flag, mlr help verb, mlr help function, and mlr help keyword. Use mlr help find ... for approximate (substring) matches, e.g. mlr help find map for all things with "map" in their names.

VERB LIST 🦋

All the verbs, in their original order — do not lose the plot:

altkv bar bootstrap bootstrap-ci case cat check clean-whitespace
count-distinct count count-similar cut decimate describe fill-down fill-empty
filter flatten format-values fraction gap grep group-by group-like gsub
having-fields head histogram json-parse json-stringify join label
latin1-to-utf8 least-frequent merge-fields most-frequent nest nothing put rank
regularize remove-empty-columns rename reorder repeat reshape sample
sec2gmtdate sec2gmt seqgen shuffle skip-trivial-records sort
sort-within-records sparkline sparsify split ssub stats1 stats2 step sub
summary surv tac tail tee template top utf8-to-latin1 unflatten uniq unspace
unsparsify

FUNCTION LIST 💜

All the functions, again exactly as listed (operators included, bestie):

abs acos acosh antimode any append apply arrayify asin asinh asserting_absent
asserting_array asserting_bool asserting_boolean asserting_bytes
asserting_empty asserting_empty_map asserting_error asserting_float
asserting_int asserting_map asserting_nonempty_map asserting_not_array
asserting_not_empty asserting_not_map asserting_not_null asserting_null
asserting_numeric asserting_present asserting_string atan atan2 atanh
base64_decode base64_encode bitcount boolean bytes capitalize cbrt ceil
clean_whitespace collapse_whitespace concat contains cos cosh count datediff
depth dhms2fsec dhms2sec distinct_count erf erfc every exec exp expm1 flatten
float floor fmtifnum fmtnum fold format fsec2dhms fsec2hms get_keys get_values
gmt2localtime gmt2nsec gmt2sec gssub gsub haskey hasvalue hex_decode
hex_encode hexfmt hms2fsec hms2sec hostname index int invqnorm is_absent
is_array is_bool is_boolean is_bytes is_empty is_empty_map is_error is_float
is_int is_map is_nan is_nonempty_map is_not_array is_not_empty is_not_map
is_not_null is_null is_numeric is_present is_string joink joinkv joinv
json_parse json_stringify kurtosis latin1_to_utf8 leafcount leftpad length
localtime2gmt localtime2nsec localtime2sec log log10 log1p logifit lstrip madd
mapdiff mapexcept mapselect mapsum max maxlen md5 mean meaneb median mexp min
minlen mmul mode msub next nsec2gmt nsec2gmtdate nsec2localdate nsec2localtime
null_count os percentile percentiles pow qnorm reduce regextract
regextract_or_else rightpad round roundm rstrip sec2dhms sec2gmt sec2gmtdate
sec2hms sec2localdate sec2localtime select sgn sha1 sha256 sha512 sin sinh
skewness sort sort_collection sparkline splita splitax splitkv splitkvx
splitnv splitnvx sqrt ssub stat stddev strfntime strfntime_local strftime
strftime_local string strip strlen strmatch strmatchx strpntime
strpntime_local strptime strptime_local sub substr substr0 substr1 sum sum2
sum3 sum4 sysntime system systime systimeint tan tanh tolower toupper truncate
typeof unflatten unformat unformatx upntime uptime urand urand32 urandelement
urandint urandrange utf8_to_latin1 variance version ! != !=~ % & && * ** + - .
.* .+ .- ./ / // < << <= <=> == =~ > >= >> >>> ?: ?? ??? ^ ^^ | || ~

💬 COMMENTS-IN-DATA FLAGS 💬

okay so bestie, Miller lets you put comments in your data, and it looks like this:

# This is a comment for a CSV file
a,b,c
1,2,3
4,5,6

Notes, and pls read these because the vibe check matters:

• Comments are only honored at the start of a line.
• In the absence of any of the below four options, comments are data like any other text. (The comments-in-data feature is opt-in.)
• When --pass-comments is used, comment lines are written to standard output immediately upon being read; they are not part of the record stream. Results may be counterintuitive. A suggestion is to place comments at the start of data files.
• --pass-comments — Immediately print commented lines (prefixed by #) within the input.
• --pass-comments-with {string} — Immediately print commented lines within input, with specified prefix. For CSV input format, the prefix must be a single character.
• --skip-comments — Ignore commented lines (prefixed by #) within the input.
• --skip-comments-with {string} — Ignore commented lines within input, with specified prefix. For CSV input format, the prefix must be a single character.

---

🗜️ COMPRESSED-DATA FLAGS 🗜️

Miller offers a few different ways to handle reading data files which have been compressed, and honestly? iconic behavior.

• Decompression done within the Miller process itself: --bz2in --gzin --zin --zstdin
• Decompression done outside the Miller process: --prepipe --prepipex

Using --prepipe and --prepipex you can specify an action to be taken on each input file. The prepipe command must be able to read from standard input; it will be invoked with {command} < {filename}. The prepipex command must take a filename as argument; it will be invoked with {command} {filename}.

Examples:

mlr --prepipe gunzip
mlr --prepipe zcat -cf
mlr --prepipe xz -cd
mlr --prepipe cat

Note that this feature is quite general and is not limited to decompression utilities. You can use it to apply per-file filters of your choice. For output compression (or other) utilities, simply pipe the output: mlr ... | {your compression command} > outputfilenamegoeshere

Lastly, note that if --prepipe or --prepipex is specified, it replaces any decisions that might have been made based on the file suffix. Likewise, --gzin/--bz2in/--zin are ignored if --prepipe is also specified.

• --bz2in — Uncompress bzip2 within the Miller process. Done by default if file ends in .bz2.
• --gzin — Uncompress gzip within the Miller process. Done by default if file ends in .gz.
• --prepipe {decompression command} — You can, of course, already do without this for single input files, e.g. gunzip < myfile.csv.gz | mlr .... Allowed at the command line, but not in .mlrrc to avoid unexpected code execution.
• --prepipe-bz2 — Same as --prepipe bz2, except this is allowed in .mlrrc.
• --prepipe-gunzip — Same as --prepipe gunzip, except this is allowed in .mlrrc.
• --prepipe-zcat — Same as --prepipe zcat, except this is allowed in .mlrrc.
• --prepipe-zstdcat — Same as --prepipe zstdcat, except this is allowed in .mlrrc.
• --prepipex {decompression command} — Like --prepipe with one exception: doesn't insert < between command and filename at runtime. Useful for some commands like unzip -qc which don't read standard input. Allowed at the command line, but not in .mlrrc to avoid unexpected code execution.
• --zin — Uncompress zlib within the Miller process. Done by default if file ends in .z.
• --zstdin — Uncompress zstd within the Miller process. Done by default if file ends in .zstd.

---

🌸 CSV/TSV-ONLY FLAGS 🌸

These are flags which are applicable to CSV format.

• --allow-ragged-csv-input, --ragged, --allow-ragged-tsv-input — If a data line has fewer fields than the header line, fill remaining keys with empty string. If a data line has more fields than the header line, use integer field labels as in the implicit-header case.
• --csv-trim-leading-space — Trims leading spaces in CSV data. Use this for data like '"foo", "bar' which is non-RFC-4180 compliant, but common.
• --headerless-csv-output, --ho, --headerless-tsv-output — Print only CSV/TSV data lines; do not print CSV/TSV header lines.
• --implicit-csv-header, --headerless-csv-input, --hi, --implicit-tsv-header — Use 1,2,3,... as field labels, rather than from line 1 of input files. Tip: combine with label to recreate missing headers.
• --lazy-quotes — Accepts quotes appearing in unquoted fields, and non-doubled quotes appearing in quoted fields.
• --no-auto-unsparsify — For CSV/TSV output: if the record keys change from one row to another, emit a blank line and a new header line. This is non-compliant with RFC 4180 but it helpful for heterogeneous data.
• --no-implicit-csv-header, --no-implicit-tsv-header — Opposite of --implicit-csv-header. This is the default anyway -- the main use is for the flags to mlr join if you have main file(s) which are headerless but you want to join in on a file which does have a CSV/TSV header. Then you could use mlr --csv --implicit-csv-header join --no-implicit-csv-header -l your-join-in-with-header.csv ... your-headerless.csv.
• --quote-all — Force double-quoting of CSV fields.
• -N — Keystroke-saver for --implicit-csv-header --headerless-csv-output.

---

🦋 DKVP-ONLY FLAGS 🦋

These are flags which are applicable to DKVP format.

• --incr-key — Without this option, keyless DKVP fields are keyed by field number. For example: a=10,b=20,30,d=40,50 is ingested as $a=10,$b=20,$3=30,$d=40,$5=50. With this option, they're keyed by a running counter of keyless fields. For example: a=10,b=20,30,d=40,50 is ingested as $a=10,$b=20,$1=30,$d=40,$2=50.

--asv or --asvlite — ASV format, bestie, for input and output 💅

--box — Unicode box-drawing tabular format for input and output 🎀

--csv or -c or --c2c — CSV format for input and output ✨

--csvlite — CSV-lite format for input and output 🌸

--dcf — Debian control file (DCF) format for input and output 🦋

--dkvp or --d2d — DKVP format for input and output 💖

--dkvpx — DKVPX format for input and output 👑

--gen-field-name — specify the field name for --igen, defaults to "i" 🎀

--gen-start — start value for --igen, defaults to 1 ⭐

--gen-step — step value for --igen, defaults to 1 🌈

--gen-stop — stop value for --igen, defaults to 100 🍓

--iasv or --iasvlite — ASV format for input data 💜

--ibox — Unicode box-drawing tabular format for input data 🎀

--icsv — CSV format for input data ✨

--icsvlite — CSV-lite format for input data 🌸

--idcf — Debian control file (DCF) format for input data 🦋

--idkvp — DKVP format for input data 💖

--igen — ignore input files and instead generate sequential numeric input using the --gen-field-name, --gen-start, --gen-step, and --gen-stop values 🧚 also lowkey peep the seqgen verb, which is more useful/intuitive 👀

--ijson — JSON format for input data 🌟

--ijsonl — JSON Lines format for input data 💫

--imd or --imarkdown — markdown-tabular format for input data 🎀

--inidx — NIDX format for input data 💅

--io {format name} — use format name for input and output data, like literally --io csv is the same as --csv ✨

--ipprint — PPRINT format for input data 🌸

--irecutils — GNU recutils (.rec) format for input data 🦋

--itsv — TSV format for input data 💖

--itsvlite — TSV-lite format for input data 🎀

--iusv or --iusvlite — USV format for input data 🌈

--ixtab — XTAB format for input data ⭐

--iyaml — YAML format for input data 🍓

--json or -j or --j2j — JSON format for input and output 👑

--jsonl or --l2l — JSON Lines format for input and output 💜

--md or --markdown — markdown-tabular format for input and output 🎀

--nidx or --n2n — NIDX format for input and output 💅

--oasv or --oasvlite — ASV format for output data ✨

--obox — Unicode box-drawing tabular format for output data 🌸

--ocsv — CSV format for output data 💖

--ocsvlite — CSV-lite format for output data 🦋

--odcf — Debian control file (DCF) format for output data 🎀

--odkvp — DKVP format for output data 🌈

--ojson — JSON format for output data ⭐

--ojsonl — JSON Lines format for output data 🍓

--omd or --omarkdown — markdown-tabular format for output data 💜

--onidx — NIDX format for output data 🎀

--opprint — PPRINT format for output data 👑

--orecutils — GNU recutils (.rec) format for output data 🦋

--otsv — TSV format for output data ✨

--otsvlite — TSV-lite format for output data 🌸

--ousv or --ousvlite — USV format for output data 💖

--oxtab — XTAB format for output data 🎀

--oyaml — YAML format for output data 🌈

--pprint or --p2p — PPRINT format for input and output ⭐

--recutils — GNU recutils (.rec) format for input and output 🍓

--tsv or -t or --t2t — TSV format for input and output 💜

--tsvlite — TSV-lite format for input and output 🎀

--usv or --usvlite — USV format for input and output 👑

--xtab or --x2x — XTAB format for input and output ✨

--xvright — right-justify values for XTAB format 💅

--yaml or --y2y — YAML format for input and output 🦋

-i {format name} — use format name for input data, like -i csv is the same as --icsv 🌸

-o {format name} — use format name for output data, like -o csv is the same as --ocsv 💖

---

🦋 FLATTEN-UNFLATTEN FLAGS 🦋

okay so these flags are the ones that tell Miller how to handle record values that are maps or arrays 💅 — specifically when your input is JSON/YAML and your output is not (that's flattening), or when your input is not JSON/YAML and your output is (that's unflattening) ✨

for the full tea, check the flatten/unflatten doc page at https://miller.readthedocs.io/en/latest/flatten-unflatten 📖

--flatsep or --jflatsep {string} — the separator used when flattening multi-level JSON keys, so like {"a":{"b":3}} becomes a:b => 3 for non-JSON formats. Defaults to . 🎀

--no-auto-flatten — when the output is not JSON or YAML, this suppresses the default auto-flatten behavior. The default is: if $y = [7,8,9] then it flattens to y.1=7,y.2=8,y.3=9, and same vibe for maps. With --no-auto-flatten, you instead get $y=[1, 2, 3] 🫶

--no-auto-unflatten — when the input is not JSON or YAML and the output is JSON or YAML, this suppresses the default auto-unflatten behavior. The default is: if the input has y.1=7,y.2=8,y.3=9 then it unflattens to $y=[7,8,9]. With --no-auto-flatten, you instead get ${y.1}=7,${y.2}=8,${y.3}=9 😭

---

🌈 FORMAT-CONVERSION KEYSTROKE-SAVER FLAGS 🌈

as keystroke-savers for format-conversion you may use the following, and honestly? iconic ✨ the letters c, t, j, l, d, n, x, p, m, and y refer to formats CSV, TSV, JSON, JSON Lines, DKVP, NIDX, XTAB, PPRINT, markdown, and YAML, respectively 💅 DCF is also supported (use --dcf for DCF in and out). GNU recutils is also supported too (use --recutils for recutils in and out) 🎀

| In\out   | CSV      | TSV      | JSON     | JSONL | DKVP  | NIDX  | XTAB  | PPRINT | Markdown | YAML   |
+----------+----------+----------+----------+-------+-------+-------+-------+--------+----------+--------+
| CSV      | --c2c,-c | --c2t    | --c2j    | --c2l | --c2d | --c2n | --c2x | --c2p  | --c2m    | --c2y  |
| TSV      | --t2c    | --t2t,-t | --t2j    | --t2l | --t2d | --t2n | --t2x | --t2p  | --t2m    | --t2y  |
| JSON     | --j2c    | --j2t    | --j2j,-j | --j2l | --j2d | --j2n | --j2x | --j2p  | --j2m    | --j2y  |
| JSONL    | --l2c    | --l2t    | --l2j    | --l2l | --l2d | --l2n | --l2x | --l2p  | --l2m    | --l2y  |
| DKVP     | --d2c    | --d2t    | --d2j    | --d2l | --d2d | --d2n | --d2x | --d2p  | --d2m    | --d2y  |
| NIDX     | --n2c    | --n2t    | --n2j    | --n2l | --n2d | --n2n | --n2x | --n2p  | --n2m    | --n2y  |
| XTAB     | --x2c    | --x2t    | --x2j    | --x2l | --x2d | --x2n | --x2x | --x2p  | --x2m    | --x2y  |
| PPRINT   | --p2c    | --p2t    | --p2j    | --p2l | --p2d | --p2n | --p2x | -p2p   | --p2m    | --p2y  |
| Markdown | --m2c    | --m2t    | --m2j    | --m2l | --m2d | --m2n | --m2x | --m2p  |          | --m2y  |
| YAML     | --y2c    | --y2t    | --y2j    | --y2l | --y2d | --y2n | --y2x | --y2p  | --y2m    | --y2y  |

and the extra keystroke-savers, because bestie deserves shortcuts 💅

• -p — keystroke-saver for --nidx --fs space --repifs ✨
• -T — keystroke-saver for --nidx --fs tab 🎀

---

⭐ JSON-ONLY FLAGS ⭐

these are the flags that are applicable to JSON output format 👑

• --jlistwrap or --jl — wrap JSON output in the outermost [ ]. This is the default for JSON output format 🦋
• --jvquoteall — force all JSON values — recursively into lists and objects — to string 💖
• --jvstack — put one key-value pair per line for JSON output (that's multi-line output). This is the default for JSON output format 🌸
• --no-jlistwrap — do not wrap JSON output in the outermost [ ]. This is the default for JSON Lines output format 💅
• --no-jvstack — put objects/arrays all on one line for JSON output. This is the default for JSON Lines output format 🌈
• --no-yarray — do not wrap YAML output in a single array document; instead emit one YAML document per record with --- between them 🎀
• --yarray or --ya — wrap YAML output in a single top-level array document. This is the default for YAML output format 👑

💀 LEGACY FLAGS 💀

Okay bestie, so these are the flags that literally do nothing in the current Miller version. Like, they're vintage. They're the low-rise jeans of flags — we keep them around so your old scripts don't break, but they're pure no-ops now. Accepted, ignored, slaying in the void. 🫶

• --jknquoteint — Type info from JSON input files is preserved throughout the processing stream now, so this one's just vibing.
• --jquoteall — Same story: type info from JSON input files is preserved throughout the processing stream now.
• --json-fatal-arrays-on-input — Miller supports arrays as of version 6, so this is a relic.
• --json-map-arrays-on-input — Miller supports arrays as of version 6, bestie.
• --json-skip-arrays-on-input — Yup: Miller supports arrays as of version 6.
• --jsonx — The --jvstack flag is now default true in Miller 6. Iconic behavior, no flag needed.
• --mmap — Miller no longer uses memory-mapping to access data files. RIP to that era.
• --no-mmap — Miller no longer uses memory-mapping to access data files, so neither of these twins does anything.
• --ojsonx — Again: the --jvstack flag is now default true in Miller 6.
• --quote-minimal — Ignored as of version 6. Types are inferred/retained through the processing flow now.
• --quote-none — Ignored as of version 6. Types are inferred/retained through the processing flow now.
• --quote-numeric — Ignored as of version 6. Types are inferred/retained through the processing flow now.
• --quote-original — Ignored as of version 6. Types are inferred/retained through the processing flow now.
• --vflatsep — Ignored as of version 6. This functionality is subsumed into JSON formatting. Like, absorbed. Gone. 💅

---

🌸 MARKDOWN-ONLY FLAGS 🌸

These ones are specifically for that markdown-tabular format moment — you know, when you want your data looking cute in a table on the internet. ✨

• --md-aligned or --markdown-aligned — Use markdown-tabular format for input and output data, with left-justified and padded columns. It implies --md, so you do not need to also pass --md. Since this sets both input and output format, it will refuse to run if an earlier flag already chose a different input or output format (e.g. --d2m) — so use --omd-aligned instead if you only want aligned markdown OUTPUT. 🎀
• --omd-aligned or --omarkdown-aligned — For markdown-tabular output, left-justify cells and pad each column to a uniform width, making the raw markdown source easier to read and maintain. (The rendered table is unaffected, bestie — this is purely for the source.) Implies --omd, so you do not need to also pass --omd. Unlike --md-aligned, this does not touch the input format, so it composes safely with e.g. --d2m regardless of flag order. Queen behavior, honestly. 👑

✨ MISCELLANEOUS FLAGS ✨

Okay bestie, these are the flags that didn't fit into any of the other little categories — the miscellaneous drawer of the Miller flag kitchen, and honestly there are some icons in here 💅

• --errors-json — Emit parse errors as a JSON object to stderr instead of a plain text message. It's intended for AI agents and scripts that branch on error kind rather than regex-matching prose. Literally equivalent to setting the MLR_ERRORS_JSON environment variable to a truthy value. robots love this one 🤖✨
• --fflush — Force buffered output to be written after every output record. The default is flush output after every record if the output is to the terminal, or less often if the output is to a file or a pipe. That default is a significant performance optimization for large files, no cap. Use this flag to force frequent updates even when output is to a pipe or file — at a performance cost, bestie, you've been warned 💀
• --files {filename} — Use this to specify a file which itself contains, one per line, names of input files. May be used more than once. a little file full of files, very inception 🌀
• --from {filename} — Use this to specify an input file before the verb(s), rather than after. May be used more than once. Example: mlr --from a.dat --from b.dat cat is the same as mlr cat a.dat b.dat. same energy, different outfit 💖
• --hash-records — This is an internal parameter which normally does not need to be modified. It controls the mechanism by which Miller accesses fields within records. In general --no-hash-records is faster, and is the default. For specific use-cases involving data having many fields, and many of them being processed during a given processing run, --hash-records might offer a slight performance benefit. niche but valid 🎀
• --infer-int-as-float or -A — Cast all integers in data files to floats. ints get the float glow-up ✨
• --infer-none or -S — Don't treat values like 123 or 456.7 in data files as int/float; leave them as strings. keep 'em as text, no mathing 🫶
• --infer-octal or -O — Treat numbers like 0123 in data files as numeric; default is string. Note that 00--07 etc scan as int; 08-09 scan as float. yes the ranges are kinda cursed, that's just how it is 😭
• --load {filename} — Load DSL script file for all put/filter operations on the command line. If the name following --load is a directory, load all *.mlr files in that directory. This is just like put -f and filter -f except it's up-front on the command line, so you can do something like alias mlr='mlr --load ~/myscripts' if you like. set it and forget it, main character alias energy 👑
• --mfrom {filenames} — Use this to specify one of more input files before the verb(s), rather than after. May be used more than once. The list of filename must end with --. This is useful for example since --from *.csv doesn't do what you might hope but --mfrom *.csv -- does. the -- is not optional bestie, it's the whole vibe 🔥
• --mload {filenames} — Like --load but works with more than one filename, e.g. --mload *.mlr --. more scripts, more slay 🦋
• --no-dedupe-field-names — By default, if an input record has a field named x and another also named x, the second will be renamed x_2, and so on. With this flag provided, the second x's value will replace the first x's value when the record is read. This flag has no effect on JSON input records, where duplicate keys always result in the last one's value being retained. last-one-wins, but only for JSON, okay? 🙈
• --no-fflush — Let buffered output not be written after every output record. The default is flush output after every record if the output is to the terminal, or less often if the output is to a file or a pipe. That default is a significant performance optimization for large files. Use this flag to allow less-frequent updates when output is to the terminal. This is unlikely to be a noticeable performance improvement, since direct-to-screen output for large files has its own overhead. tl;dr: don't expect much 💀
• --no-hash-records — See --hash-records. yes queen, go read the other one ⬆️
• --no-shell — Disable Miller's ability to run external commands: the DSL system and exec functions, piped redirects such as tee | "command", and --prepipe/--prepipex all fail cleanly instead of executing. Equivalent to setting the MLR_NO_SHELL environment variable to a truthy value. It's intended for running agent-constructed command lines (e.g. via mlr mcp) without also granting arbitrary command execution. Once disabled, shell-outs cannot be re-enabled for the rest of the process. no take-backsies, periodt 🔒
• --norc — Do not load a .mlrrc file. skip the config, rawdog the defaults 😌
• --nr-progress-mod {m} — With m a positive integer: print filename and record count to os.Stderr every m input records. little progress check-ins so you know it's alive 🍓
• --ofmt {format} — E.g. %.18f, %.0f, %9.6e. Please use sprintf-style codes (https://pkg.go.dev/fmt) for floating-point numbers. If not specified, default formatting is used. See also the fmtnum function and the format-values verb. make your floats cute and specific 💅
• --ofmte {n} — Use --ofmte 6 as shorthand for --ofmt %.6e, etc. short and sweet ⭐
• --ofmtf {n} — Use --ofmtf 6 as shorthand for --ofmt %.6f, etc. same but with an f 🎀
• --ofmtg {n} — Use --ofmtg 6 as shorthand for --ofmt %.6g, etc. and a g, the trifecta 🌈
• --profile or -P {name} — Apply the settings from the [name] section of your .mlrrc file, after any global (pre-section) settings. It's an error if no such section exists in any .mlrrc file processed. For more information please see https://miller.readthedocs.io/en/latest/customization/. profile pic but for your settings 💖
• --records-per-batch {n} — This is an internal parameter for maximum number of records in a batch size. Normally this does not need to be modified, except when input is from tail -f. See also https://miller.readthedocs.io/en/latest/reference-main-flag-list/. internal knob, hands off unless you know 🫶
• --s-no-comment-strip {file name} — Take command-line flags from file name, like -s, but with no comment-stripping. For more information please see https://miller.readthedocs.io/en/latest/scripting/. comments stay, nothing gets yeeted 🌸
• --seed {n} — with n of the form 12345678 or 0xcafefeed. For put/filter urand, urandint, and urand32. deterministic randomness, iconic ✨
• --tz {timezone} — Specify timezone, overriding $TZ environment variable (if any). your timezone era, your rules 🕰️
• -I — Process files in-place. For each file name on the command line, output is written to a temp file in the same directory, which is then renamed over the original. Each file is processed in isolation: if the output format is CSV, CSV headers will be present in each output file, statistics are only over each file's own records; and so on. glow-up in place, no leftovers 💅
• -n — Process no input files, nor standard input either. Useful for mlr put with begin/end statements only. (Same as --from /dev/null.) Also useful in mlr -n put -v '...' for analyzing abstract syntax trees (if that's your thing). no input, all vibes 🌌
• -s {file name} — Take command-line flags from file name. For more information please see https://miller.readthedocs.io/en/latest/scripting/. your flags, but in a file 🎀
• -x — If any record has an error value in it, report it and stop the process. The default is to print the field value as (error) and continue. strict mode activated, no cap 💀

---

🌈 OUTPUT-COLORIZATION FLAGS 🌈

Miller literally uses color to make output pop — like, it highlights things for you. And bestie, you get to pick the colors. ✨

Heads up: output colorization does not work on Windows. 😭

🎀 The things that get colored

• Keys in CSV header lines, JSON keys, etc.
• Values in CSV data lines, JSON scalar values, etc. in regression-test output
• Some online-help strings

🎀 The rules for coloring

• By default, Miller colorizes output only if writing to stdout and stdout is a TTY.
  • Example — color: mlr --csv cat foo.csv
  • Example — no color: mlr --csv cat foo.csv > bar.csv
  • Example — no color: mlr --csv cat foo.csv | less
• The default colors were chosen since they look OK with white or black terminal background, and are differentiable with common varieties of human color vision. (Thoughtful king behavior 👑)

🎀 The mechanisms behind the coloring

• Miller uses ANSI escape sequences only. This does not work on Windows except within Cygwin.
• Requires TERM environment variable to be set to non-empty string.
• Doesn't try to check to see whether the terminal is capable of 256-color ANSI vs 16-color ANSI. Note that if colors are in the range 0..15 then 16-color ANSI escapes are used, so this is in the user's control.

🎀 How you can control the colorization

• Suppression/unsuppression:
  • Environment variable export MLR_NO_COLOR=true or export NO_COLOR=true means don't color even if stdout+TTY.
  • Environment variable export MLR_ALWAYS_COLOR=true means do color even if not stdout+TTY. For example, you might want to use this when piping mlr output to less -r.
  • Command-line flags --no-color or -M, --always-color or -C.
• Color choices can be specified by using environment variables, or command-line flags, with values 0..255:
  • export MLR_KEY_COLOR=208, MLR_VALUE_COLOR=33, etc.: MLR_KEY_COLOR MLR_VALUE_COLOR MLR_PASS_COLOR MLR_FAIL_COLOR MLR_REPL_PS1_COLOR MLR_REPL_PS2_COLOR MLR_HELP_COLOR
  • Command-line flags --key-color 208, --value-color 33, etc.: --key-color --value-color --pass-color --fail-color --repl-ps1-color --repl-ps2-color --help-color
  • This is particularly useful if your terminal's background color clashes with current settings. (we've all been there 💀)

If environment-variable settings and command-line flags are both provided, the latter take precedence. Flags stay winning, fr fr.

Colors can be specified using names such as "red" or "orchid": please see mlr --list-color-names to see available names. They can also be specified using numbers in the range 0..255, like 170: please see mlr --list-color-codes. You can also use "bold", "underline", and/or "reverse". Additionally, combinations of those can be joined with a -, like "red-bold", "bold-170", "bold-underline", etc. Mix and match, it's giving custom aesthetic 🦋

---

• --always-color or -C — Instructs Miller to colorize output even when it normally would not. Useful for piping output to less -r.
• --fail-color — Specify the color (see --list-color-codes and --list-color-names) for failing cases in mlr regtest.
• --help-color — Specify the color (see --list-color-codes and --list-color-names) for highlights in mlr help output.
• --key-color — Specify the color (see --list-color-codes and --list-color-names) for record keys.
• --list-color-codes — Show the available color codes in the range 0..255, such as 170 for example.
• --list-color-names — Show the names for the available color codes, such as orchid for example.
• --no-color or -M — Instructs Miller to not colorize any output.
• --pass-color — Specify the color (see --list-color-codes and --list-color-names) for passing cases in mlr regtest.
• --value-color — Specify the color (see --list-color-codes and --list-color-names) for record values.

---

✨ PPRINT-ONLY FLAGS ✨

These are flags which are applicable to PPRINT format. PPRINT girlies, this one's for you 🎀

• --barred or --barred-output — Prints a border around PPRINT output. (aesthetics unlocked 🔥)
• --barred-input — When used in conjunction with --pprint, accepts barred input.
• --barred-unicode — Uses unicode printing chars for barred output.
• --fixed {string} — Fixed width specification. One of 'widths:<col1-width>,<col2-width>,...', left-align, left-align-multi-word, right-align, right-align-multi-word
• --fw {string} — Shortcut for --fixed left-align-multi-word
• --right — Right-justifies all fields for PPRINT output.
• --right-align-numeric — Right-justifies fields with numeric values for PPRINT output, leaving other fields left-justified. Headers are right-justified over columns whose values are all numeric, so that header and data share the same alignment. Also applies to markdown output, where numeric columns get right-alignment markers (---:) in the header-separator line.

---

💜 PROFILING FLAGS 💜

These are flags for profiling Miller performance. For the girlies who need to know why it's slow 👀

• --cpuprofile {CPU-profile file name} — Create a CPU-profile file for performance analysis. Instructions will be printed to stderr. This flag must be the very first thing after mlr on the command line. (like, first, no exceptions 🫶)
• --time — Print elapsed execution time in seconds to stderr at the end of the execution of the program.
• --traceprofile — Create a trace-profile file for performance analysis. Instructions will be printed to stderr. This flag must be the very first thing after mlr on the command line.

💅 SEPARATOR FLAGS 💅

okay so this is the section where we talk about the literal glue between your data pieces 🫶. Peep the Separators doc page for the full tea on record separators, field separators, and pair separators. Also go look at the File formats doc page, or just run mlr help file-formats, for more about the file formats Miller supports.

Real quick, the short version:

• For DKVP records like x=1,y=2,z=3, the fields are separated by a comma, the key-value pairs are separated by a comma, and each record is separated from the next by a newline.
• Each file format has its own default separators.
• Most formats, such as CSV, don't support pair-separators: keys are on the CSV header line and values are on each CSV data line; keys and values are not placed next to one another.
• Some separators are not programmable: for example JSON uses a colon as a pair separator but this is non-modifiable in the JSON spec.
• You can set separators differently between Miller's input and output — hence --ifs and --ofs, etc. 💖

---

🌸 Notes about line endings

• Default line endings (--irs and --ors) are newline which is interpreted to accept carriage-return/newline files (e.g. on Windows) for input, and to produce platform-appropriate line endings on output.
• For CSV, CSV-lite, TSV, and TSV-lite output, ORS may be either newline (the default) or carriage-return/newline: e.g. --ors crlf or --ors '\r\n' for RFC-4180-style line endings on any platform.

---

🦋 Notes about all other separators

• IPS/OPS are only used for DKVP and XTAB formats, since only in these formats do key-value pairs appear juxtaposed.
• IRS/ORS are ignored for XTAB format. Nominally IFS and OFS are newlines; XTAB records are separated by two or more consecutive IFS/OFS — i.e. a blank line. Everything above about --irs/--ors/--rs auto becomes --ifs/--ofs/--fs auto for XTAB format. (XTAB's default IFS/OFS are "auto".)
• OFS must be single-character for PPRINT format. This is because it is used with repetition for alignment; multi-character separators would make alignment impossible.
• OPS may be multi-character for XTAB format, in which case alignment is disabled.
• FS/PS are ignored for markdown format; RS is used.
• All FS and PS options are ignored for JSON format, since they are not relevant to the JSON format.

---

🎀 you can specify separators in any of these ways (examples, bestie)

• Type them out, quoting as necessary for shell escapes, e.g. --fs '|' --ips :
• C-style escape sequences, e.g. --rs '\r\n' --fs '\t'.
• To avoid backslashing, you can use any of the following names:

ascii_esc  = "\x1b"
ascii_etx  = "\x03"
ascii_fs   = "\x1c"
ascii_gs   = "\x1d"
ascii_null = "\x00"
ascii_rs   = "\x1e"
ascii_soh  = "\x01"
ascii_stx  = "\x02"
ascii_us   = "\x1f"
asv_fs     = "\x1f"
asv_rs     = "\x1e"
colon      = ":"
comma      = ","
cr         = "\r"
crcr       = "\r\r"
crlf       = "\r\n"
crlfcrlf   = "\r\n\r\n"
equals     = "="
lf         = "\n"
lflf       = "\n\n"
newline    = "\n"
pipe       = "|"
semicolon  = ";"
slash      = "/"
space      = " "
tab        = "\t"
usv_fs     = "\xe2\x90\x9f"
usv_rs     = "\xe2\x90\x9e"

• Similarly, you can use the following for --ifs-regex and --ips-regex:

spaces     = "( )+"
tabs       = "(\t)+"
whitespace = "([ \t])+"

---

👑 default separators by format

Format   FS     PS     RS
box      " "    N/A    "\n"
csv      ","    N/A    "\n"
csvlite  ","    N/A    "\n"
dcf      N/A    N/A    N/A
dkvp     ","    "="    "\n"
dkvpx    ","    "="    "\n"
gen      ","    N/A    "\n"
json     N/A    N/A    N/A
markdown " "    N/A    "\n"
nidx     " "    N/A    "\n"
pprint   " "    N/A    "\n"
recutils N/A    N/A    N/A
tsv      "    "    N/A    "\n"
xtab     "\n"   " "    "\n\n"
yaml     N/A    N/A    N/A

---

✨ the actual flags ✨

• --fs {string} — Specify FS for input and output.
• --ifs {string} — Specify FS for input.
• --ifs-regex {string} — Specify FS for input as a regular expression.
• --ips {string} — Specify PS for input.
• --ips-regex {string} — Specify PS for input as a regular expression.
• --irs {string} — Specify RS for input.
• --ofs {string} — Specify FS for output.
• --ops {string} — Specify PS for output.
• --ors {string} — Specify RS for output.
• --ps {string} — Specify PS for input and output.
• --repifs — Let IFS be repeated: e.g. for splitting on multiple spaces.
• --rs {string} — Specify RS for input and output.

---

🧰 AUXILIARY COMMANDS 🧰

Available entries, bestie:

mlr aux-list
mlr hex
mlr lecat
mlr termcvt
mlr unhex

For more information, please invoke mlr {subcommand} --help. 💜

---

🎀 MLRRC 🎀

no cap, this is the customization bit and it's lowkey iconic. You can set up personal defaults via a $HOME/.mlrrc and/or ./.mlrrc. For example, if you usually process CSV, then you can put --csv in your .mlrrc file and that will be the default input/output format unless otherwise specified on the command line. ✨

The .mlrrc file format is one --flag or --option value per line, with the leading -- optional. Hash-style comments and blank lines are ignored.

Sample .mlrrc:

# Input and output formats are CSV by default (unless otherwise specified
# on the mlr command line):
csv
# These are no-ops for CSV, but when I do use JSON output, I want these
# pretty-printing options to be used:
jvstack
jlistwrap

💅 How to specify location of .mlrrc

• If $MLRRC is set:
  • If its value is "__none__" then no .mlrrc files are processed.
  • Otherwise, its value (as a filename) is loaded and processed. If there are syntax errors, they abort mlr with a usage message (as if you had mistyped something on the command line). If the file can't be loaded at all, though, it is silently skipped.
  • Any .mlrrc in your home directory or current directory is ignored whenever $MLRRC is set in the environment.
• Otherwise:
  • If $HOME/.mlrrc exists, it's then processed as above.
  • If ./.mlrrc exists, it's then also processed as above.
  • (I.e. current-directory .mlrrc defaults are stacked over home-directory .mlrrc defaults.)
• The command-line flag --norc can be used to suppress loading the .mlrrc file even when other conditions are met.

See also: https://miller.readthedocs.io/en/latest/customization.html

---

🌀 REPL 🌀

the interactive era 😭💖

Usage: mlr repl [options] {zero or more data-file names}

• -v — Prints the expressions's AST (abstract syntax tree), which gives full transparency on the precedence and associativity rules of Miller's grammar, to stdout.
• -d — Like -v but uses a parenthesized-expression format for the AST.
• -D — Like -d but with output all on one line.
• -w — Show warnings about uninitialized variables
• -q — Don't show startup banner
• -s — Don't show prompts
• --load {DSL script file} — Load script file before presenting the prompt. If the name following --load is a directory, load all "*.mlr" files in that directory.
• --mload {DSL script files} — Like --load but works with more than one filename, e.g. '--mload *.mlr --'.
• -h|--help — Show this message.

Or any --icsv, --ojson, etc. reader/writer options as for the main Miller command line.

Any data-file names are opened just as if you had waited and typed :open {filenames} at the Miller REPL prompt.

---

⚡ VERBS ⚡

🍓 altkv

Usage: mlr altkv [options]

Given fields with values of the form a,b,c,d,e,f emits a=b,c=d,e=f pairs. slay.

Options:

• -h|--help — Show this message.

---

📊 bar

Usage: mlr bar [options]

Replaces a numeric field with a number of asterisks, allowing for cheesy bar plots. These align best with --opprint or --oxtab output format. iykyk 🎀

Options:

• -f {a,b,c} — Field names to convert to bars.
• --lo {lo} — Lower-limit value for min-width bar: default '0.000000'.
• --hi {hi} — Upper-limit value for max-width bar: default '100.000000'.
• -w {n} — Bar-field width: default '40'.
• --auto — Automatically computes limits, ignoring --lo and --hi. Holds all records in memory before producing any output.
• -c {character} — Fill character: default '*'.
• -x {character} — Out-of-bounds character: default '#'.
• -b {character} — Blank character: default '.'.
• -h|--help — Show this message.

Heads up bestie: nominally the fill, out-of-bounds, and blank characters will be strings of length 1. However you can make them all longer if you so desire. 💅

---

🎲 bootstrap

Usage: mlr bootstrap [options]

Emits an n-sample, with replacement, of the input records. See also mlr sample and mlr shuffle. ⭐

Options:

• -n {n} — Number of samples to output. Defaults to number of input records. Must be non-negative.
• -h|--help — Show this message.

---

🔥 bootstrap-ci

Usage: mlr bootstrap-ci [options]

Computes bootstrap confidence intervals for statistics of given fields, accumulated across the input record stream: values are resampled with replacement many times, the statistic is computed on each resample, and the confidence interval is taken from percentiles of the resampled statistics. For each value field and statistic, outputs the full-data statistic in {field}_{stat}, along with confidence-interval endpoints in {field}_{stat}_lo and {field}_{stat}_hi. Use mlr --seed for reproducible results. See also mlr bootstrap and mlr stats1. fr fr, this one's the stats girlie. 💜

Options:

• -a {mean,...} — Names of statistics to bootstrap: one or more of the listed values, as in mlr stats1 -a. Also accepts median (same as p50) and percentiles p{n} for n in 0..100. Defaults to mean.
• -f {a,b,c} — Value-field names on which to compute statistics. Required.
• -g {d,e,f} — Optional group-by-field names.
• -n {n} — Number of bootstrap resamples. Must be positive. Defaults to 1000.
• -c {level} — Confidence level, strictly between 0 and 1. Defaults to 0.95.
• -i — Use interpolated percentiles, like R's type=7, for percentile statistics as well as for the confidence-interval endpoints; default like type=1.
• -h|--help — Show this message.

Example:

mlr --seed 12345 bootstrap-ci -f x,y

Example:

mlr --seed 12345 bootstrap-ci -a mean,median -f x -g shape -n 5000 -c 0.99

🌸 case 🌸

So you want to change the casing of your keys and/or values? Say less bestie, this verb ate.

Usage: mlr case [options]

It uppercases strings in record keys and/or values. That's the whole gig. 💅

Options:

• -k — case only keys, not keys and values
• -v — case only values, not keys and values
• -f {a,b,c} — specify which field names to case (default: all)
• -u — convert to uppercase
• -l — convert to lowercase
• -s — convert to sentence case (capitalize first letter)
• -t — convert to title case (capitalize words)
• -h, --help — show this message

---

🐱 cat 🐱

The most lowkey iconic verb. It literally just passes input records directly to output — most useful for format conversion. No thoughts, head empty, records in, records out. ✨

Usage: mlr cat [options]

Options:

• -n — prepend field n to each record with record-counter starting at 1
• -N {name} — prepend field {name} to each record with record-counter starting at 1
• -g {a,b,c} — optional group-by-field names for counters, e.g. a,b,c
• --filename — prepend current filename to each record
• --filenum — prepend current filenum (1-up) to each record
• -h, --help — show this message

---

🎀 check 🎀

Vibe-check your data, bestie. This one consumes records without printing any output — the one exception being that warnings get printed to stderr. Super useful for doing a well-formatted check on input data. 🙈

Usage: mlr check [options]

Current checks are:

• data are parseable
• if any key is the empty string

Options:

• -h, --help — show this message

---

🧚 clean-whitespace 🧚

For each record, for each field in the record, this whitespace-cleans the keys and/or values. Whitespace-cleaning entails stripping leading and trailing whitespace, and replacing multiple whitespace with singles. It's giving spa day for your strings. 🫧

For finer-grained control, please see the DSL functions lstrip, rstrip, strip, collapse_whitespace, and clean_whitespace.

Usage: mlr clean-whitespace [options]

Options:

• -k, --keys-only — do not touch values
• -v, --values-only — do not touch keys
• -h, --help — show this message

⚠️ It is an error to specify -k as well as -v — to clean keys and values, leave off -k as well as -v. Like, pick a lane. 😭

---

🔥 count-distinct 🔥

Prints number of records having distinct values for specified field names. Same as uniq -c. Fr fr, that's it.

Usage: mlr count-distinct [options]

Options:

• -f {a,b,c} — field names for distinct count (synonym for -g)
• -g {a,b,c} — field names for distinct count
• -x {a,b,c} — field names to exclude for distinct count; use each record's other fields instead
• -n — show only the number of distinct values. Not compatible with -u.
• -o {name} — field name for output count. Default "count". Ignored with -u.
• -u — do unlashed counts for multiple field names. With -f a,b and without -u, computes counts for distinct combinations of a and b field values. With -f a,b and with -u, computes counts for distinct a field values and counts for distinct b field values separately.
• -h, --help — show this message

---

👑 count 👑

Prints number of records, optionally grouped by distinct values for specified field names. The main character of counting verbs. 💜

Usage: mlr count [options]

Options:

• -g {a,b,c} — optional group-by-field names for counts, e.g. a,b,c
• -n — show only the number of distinct values. Not interesting without -g.
• -o {name} — field name for output-count. Default "count".
• -h, --help — show this message

---

💀 count-similar 💀

Ingests all records, then emits each record augmented by a count of the number of other records having the same group-by field values. So it's like, "how many of u are there?" for every single record. 🫶

Usage: mlr count-similar [options]

Options:

• -g {a,b,c} — group-by-field names for counts, e.g. a,b,c
• -o {name} — field name for output-counts. Defaults to "count".
• -h, --help — show this message

---

✂️ cut ✂️

Passes through input records with specified fields included/excluded. Not me carefully curating my fields. 🎀

Usage: mlr cut [options]

Options:

• -f {a,b,c} — comma-separated field names to include or exclude, e.g. a,b,c.
• -o — retain fields in the order specified by -f rather than in input-record order.
• -x, --complement — exclude, rather than include, the field names specified by -f.
• -r — treat field names as regular expressions. "ab", "a.*b" will match any field name containing the substring "ab" or matching "a.*b", respectively; anchors of the form "^ab$", "^a.*b$" may be used.
• -h, --help — show this message

Examples:

mlr cut -f hostname,status
mlr cut -x -f hostname,status
mlr cut -r -f '^status$,sda[0-9]'
mlr cut -r -f '^status$,"sda[0-9]"'
mlr cut -r -f '^status$,"sda[0-9]"i' (this is case-insensitive)

---

🍓 decimate 🍓

Passes through one of every n records, optionally by category. Decimation, but make it cute. ✨

Usage: mlr decimate [options]

Options:

• -b — decimate by printing first of every n
• -e — decimate by printing last of every n (default)
• -g {a,b,c} — optional group-by-field names for decimate counts, e.g. a,b,c.
• -n {n} — decimation factor (default 10).
• -h, --help — show this message

---

🦋 describe 🦋

Shows a compact schema for the input data: field names, types, and value shape. It emits one output record per input field, with types seen, counts, cardinality, min/max, and (for low-cardinality fields) the complete set of distinct values. It's giving "tell me everything about u" energy. 🔍

Usage: mlr describe [options]

Output fields, one record per input field:

• field_name — name of the input field
• types — map from type name (int, string, etc.) to occurrence count
• count — number of records in which the field appears
• null_count — count of field values either empty string or JSON null
• distinct_count — count of distinct values for the field
• min, max — minimum/maximum field value (works for strings as well as numbers)
• values — all distinct values, in order first seen — only for fields whose distinct_count is within the -n limit

Notes:

• Distinctness is computed on string representations — so 4.1 and 4.10 are counted as distinct here. No cap, they're different strings bestie. 💅
• Use mlr --ojson describe for a machine-readable JSON document; in tabular output formats the types map and values array are flattened.
• See also the summary verb, which reports summary statistics (mean, percentiles, etc.).

Options:

• -n, --max-values {n} — list a field's distinct values only if it has at most {n} of them; 0 suppresses the values array entirely. Defaults to 20.
• -h, --help — show this message

---

🌈 fill-down 🌈

If a given record has a missing value for a given field, fill that from the corresponding value from a previous record, if any. Basically "borrow from the last person who had their life together." 🫶

By default, a 'missing' field either is absent, or has the empty-string value. With -a, a field is 'missing' only if it is absent.

Usage: mlr fill-down [options]

Options:

• --all — operate on all fields in the input.
• -a, --only-if-absent — if a given record has a missing value for a given field, fill that from the corresponding value from a previous record, if any. By default, a 'missing' field either is absent, or has the empty-string value. With -a, a field is 'missing' only if it is absent.
• -f {a,b,c} — field names for fill-down.
• -h, --help — show this message

---

🎀 fill-empty 🎀

Fills empty-string fields with specified fill-value. The glow-up for your blank cells. ✨

Usage: mlr fill-empty [options]

Options:

• -v {string} — fill-value: defaults to "N/A"
• -S — don't infer type — so -v 0 would fill string 0 not int 0.
• -h, --help — show this message

---

💖 filter 💖

Usage: mlr filter [options] {DSL expression}

Lets you use a domain-specific language to programmatically filter which stream records will be output. This is the moment where you become a literal wizard, bestie. 🧙‍♀️✨

See also: https://miller.readthedocs.io/en/latest/reference-verbs

Options:

• -f {file name} — file containing a DSL expression (see examples below). If the filename is a directory, all *.mlr files in that directory are loaded.
• -e {expression} — DSL expression to evaluate. You can use this after -f to add an expression. Example use case: define functions/subroutines in a file you specify with -f, then call them with an expression you specify with -e.
• -s {name=value} — predefines out-of-stream variable @name to have the given value. Thus mlr put -s foo=97 '$column += @foo' is like mlr put 'begin {@foo = 97} $column += @foo'. The value part is subject to type-inferencing. May be specified more than once, e.g. -s name1=value1 -s name2=value2. Note: the value may be an environment variable, e.g. -s sequence=$SEQUENCE.
• -x — prints records for which {expression} evaluates to false, not true, i.e. invert the sense of the filter expression. Default false.
• -q — does not include the modified record in the output stream. Useful for when all desired output is in begin and/or end blocks.
• -S — no-op in Miller 6 and above, since type-inferencing is now done by the record-readers before filter/put is executed. Supported as a no-op pass-through flag for backward compatibility.
• -F — no-op in Miller 6 and above, since type-inferencing is now done by the record-readers before filter/put is executed. Supported as a no-op pass-through flag for backward compatibility.
• -w — print warnings about things like uninitialized variables.
• -W — same as -w, but exit the process if there are any warnings.
• -p — prints the expression's AST (abstract syntax tree), which gives full transparency on the precedence and associativity rules of Miller's grammar, to stdout.
• -d — like -p but uses a parenthesized-expression format for the AST.
• -D — like -d but with output all on one line.
• -E — echo DSL expression before printing parse-tree.
• -v — same as -E -p.
• -X — exit after parsing but before stream-processing. Useful with -v/-d/-D, if you only want to look at parser information.
• --explain — parse and type-check the DSL expression, report whether it is valid, and exit without reading the input stream. Exit status is 0 if the expression is valid and non-zero otherwise; combine with --errors-json for a machine-readable error.
• -h, --help — show this message

💡 If you mix -e and -f then the expressions are evaluated in the order encountered. Since the expression pieces are simply concatenated, please be sure to use intervening semicolons to separate expressions. Semicolons are the glue holding your whole situation together, iykyk.

Parser-info options are -w, -W, -p, -d, -D, -E, -v, and -X.

Records will pass the filter depending on the last bare-boolean statement in the DSL expression. That can be the result of <, ==, >, etc., the return value of a function call which returns boolean, etc.

Some examples, bestie:

mlr --csv --from example.csv filter '$color == "red"'
mlr --csv --from example.csv filter '$color == "red" && flag == true'

More example filter expressions, because I know you love a little inspo:

• First record in each file:

'FNR == 1'

• Subsampling:

'urand() < 0.001'

• Compound booleans:

'$color != "blue" && $value > 4.2'
'($x < 0.5 && $y < 0.5) || ($x > 0.5 && $y > 0.5)'

• Regexes with case-insensitive flag:

'($name =~ "^sys.*east$") || ($name =~ "^dev.[0-9]+"i)'

• Assignments, then bare-boolean filter statement:

'$ab = $a+$b; $cd = $c+$d; $ab != $cd'

• Bare-boolean filter statement within a conditional:

'if (NR < 100) {
  $x > 0.3;
} else {
  $x > 0.002;
}
'

• Using 'any' higher-order function to see if $index is 10, 20, or 30:

'any([10,20,30], func(e) {return $index == e})'

See also https://miller.readthedocs.io/reference-dsl for more context. 🫶

---

🦋 flatten

Usage:

mlr flatten [options]

Flattens multi-level maps to single-level ones. Example: field with name 'a' and value '{"b": { "c": 4 }}' becomes name 'a.b.c' and value 4. Squish squish, no more nesting ✨

Options:

• -f — Comma-separated list of field names to flatten (default all).
• -s {string} — Separator, defaulting to mlr --flatsep value.
• -h|--help — Show this message.

---

🎀 format-values

Usage:

mlr format-values [options]

Applies format strings to all field values, depending on autodetected type. Here's the tea:

• If a field value is detected to be integer, applies integer format.
• Else, if a field value is detected to be float, applies float format.
• Else, applies string format.

Note: this is a low-keystroke way to apply formatting to many fields. To get finer control, please see the fmtnum function within the mlr put DSL.

Note: this verb lets you apply arbitrary format strings, which can produce undefined behavior and/or program crashes. See your system's "man printf". No cap, be careful out there 😭

Options:

• -i {integer format} — Integer format string; defaults to "%d". Examples: "%06lld", "%08llx". Note that Miller integers are long long so you must use formats which apply to long long, e.g. with ll in them. Undefined behavior results otherwise.
• -f {float format} — Float format string; defaults to "%f". Examples: "%8.3lf", "%.6le". Note that Miller floats are double-precision so you must use formats which apply to double, e.g. with l[efg] in them. Undefined behavior results otherwise.
• -s {string format} — String format string; defaults to "%s". Examples: "_%s", "%08s". Note that you must use formats which apply to string, e.g. with s in them. Undefined behavior results otherwise.
• -n — Coerce field values autodetected as int to float, and then apply the float format.
• -h|--help — Show this message.

---

💜 fraction

Usage:

mlr fraction [options]

For each record's value in specified fields, computes the ratio of that value to the sum of values in that field over all input records. E.g. with input records x=1 x=2 x=3 and x=4, emits output records x=1,x_fraction=0.1 x=2,x_fraction=0.2 x=3,x_fraction=0.3 and x=4,x_fraction=0.4. Math girl summer 💅

Note: this is internally a two-pass algorithm: on the first pass it retains input records and accumulates sums; on the second pass it computes quotients and emits output records. This means it produces no output until all input is read. So yeah, it's a patient verb, lowkey.

Options:

• -f {a,b,c} — Field name(s) for fraction calculation
• -g {d,e,f} — Optional group-by-field name(s) for fraction counts
• -p — Produce percents [0..100], not fractions [0..1]. Output field names end with "_percent" rather than "_fraction"
• -c — Produce cumulative distributions, i.e. running sums: each output value folds in the sum of the previous for the specified group. E.g. with input records x=1 x=2 x=3 and x=4, emits output records x=1,x_cumulative_fraction=0.1 x=2,x_cumulative_fraction=0.3 x=3,x_cumulative_fraction=0.6 and x=4,x_cumulative_fraction=1.0
• -h|--help — Show this message.

---

🦋 gap

Usage:

mlr gap [options]

Emits an empty record every n records, or when certain values change. One of -n or -g is required — like, actually required, bestie, don't skip it 🙈

Options:

• -g {a,b,c} — Print a gap whenever values of these fields (e.g. a,b,c) changes.
• -n {n} — Print a gap every n records. Ignored if -g is present.
• -h|--help — Show this message.

---

🔍 grep

Usage:

mlr grep [options] {regular expression}

Passes through records which match the regular expression.

Options:

• -i — Use case-insensitive search.
• -v — Invert: pass through records which do not match the regex.
• -a — Only grep for values, not keys and values.
• -h|--help — Show this message.

Note that "mlr filter" is more powerful, but requires you to know field names. By contrast, "mlr grep" allows you to regex-match the entire record. It does this by formatting each record in memory as DKVP (or NIDX, if -a is supplied), using OFS "," and OPS "=", and matching the resulting line against the regex specified here. In particular, the regex is not applied to the input stream: if you have CSV with header line "x,y,z" and data line "1,2,3" then the regex will be matched, not against either of these lines, but against the DKVP line "x=1,y=2,z=3". Furthermore, not all the options to system grep are supported, and this command is intended to be merely a keystroke-saver. To get all the features of system grep, you can do

"mlr --odkvp ... | grep ... | mlr --idkvp ..."

iykyk 🧚

---

🌸 group-by

Usage:

mlr group-by [options] {comma-separated field names}

Outputs records in batches having identical values at specified field names. Grouping them up like a little friend squad 💖

Options:

• -h|--help — Show this message.

---

✨ group-like

Usage:

mlr group-like [options]

Outputs records in batches having identical field names. Same keys, same vibes, all batching together.

Options:

• -h|--help — Show this message.

---

🎀 gsub

Usage:

mlr gsub [options]

Replaces old string with new string in specified field(s), with regex support for the old string and handling multiple matches, like the gsub DSL function. The replacement string supports C-style backslash escapes such as \n, \t, and \x1f. Write \\ to get a literal backslash.

See also the sub and ssub verbs.

Options:

• -f {a,b,c} — Field names to apply substitution to.
• -r {regex} — Regular expression for field names to apply substitution to.
• -a — Apply substitution to all fields.
• -h|--help — Show this message.

---

👑 having-fields

Usage:

mlr having-fields [options]

Conditionally passes through records depending on each record's field names.

Options:

• --at-least {comma-separated names} — Pass records that have at least these field names.
• --which-are {comma-separated names} — Pass records whose field names are exactly these.
• --at-most {comma-separated names} — Pass records that have at most these field names.
• --all-matching {regular expression} — Pass records where all field names match the regex.
• --any-matching {regular expression} — Pass records where any field name matches the regex.
• --none-matching {regular expression} — Pass records where no field name matches the regex.
• -h|--help — Show this message.

Examples, because examples eat:

mlr having-fields --which-are amount,status,owner
mlr having-fields --any-matching 'sda[0-9]'
mlr having-fields --any-matching '"sda[0-9]"'
mlr having-fields --any-matching '"sda[0-9]"i' (this is case-insensitive)

---

🍓 head

Usage:

mlr head [options]

Passes through the first n records, optionally by category. Without -g, ceases consuming more input (i.e. is fast) when n records have been read. Early exit queen ⚡

Options:

• -g {a,b,c} — Optional group-by-field names for head counts, e.g. a,b,c.
• -n {n} — Head-count to print. Default 10. A negative count, e.g. -n -2, passes through all but the last n records, optionally by category.
• -h|--help — Show this message.

---

📊 histogram

Just a histogram. Input values < lo or > hi are not counted.

Usage:

mlr histogram [options]

Options:

• -f {a,b,c} — Value-field names for histogram counts.
• --lo {lo} — Histogram low value.
• --hi {hi} — Histogram high value.
• --nbins {n} — Number of histogram bins. Defaults to 20.
• --auto — Automatically computes limits, ignoring --lo and --hi. Holds all values in memory before producing any output.
• -o {prefix} — Prefix for output field name. Default: no prefix.
• -s — Print a one-line Unicode sparkline per field instead of per-bin counts.
• -h|--help — Show this message.

With -s, output is one record per value-field, with a sparkline field instead of one record per bin. Sparklines are literally so cute, obsessed 🌈

---

🧚 json-parse

Usage:

mlr json-parse [options]

Tries to convert string field values to parsed JSON, e.g. "[1,2,3]" -> [1,2,3].

Options:

• -f {a,b,c} — Comma-separated list of field names to json-parse (default all).
• -k — If supplied, then on parse fail for any cell, keep the (unparsable) input value for the cell.
• -h|--help — Show this message.

---

💅 json-stringify

Usage:

mlr json-stringify [options]

Produces string field values from field-value data, e.g. [1,2,3] -> "[1,2,3]".

Options:

• -f {a,b,c} — Comma-separated list of field names to json-stringify (default all).
• --jvstack — Produce multi-line JSON output.
• --no-jvstack — Produce single-line JSON output per record (default).
• -h|--help — Show this message.

join 💍

Usage: mlr join [options]

Okay so this one glues records from a left file you name with records from all the file names chilling at the end of the Miller argument list. Functionality is basically the same as the system join command, except it's for record streams. Slay.

Options:

• -f {left file name} — the left file name for the join. That's the one, bestie.
• -j {a,b,c} — comma-separated join-field names for the output.
• -l {a,b,c} — comma-separated join-field names for the left input file; defaults to the -j values if you omit it.
• -r {a,b,c} — comma-separated join-field names for the right input file(s); also defaults to the -j values if omitted.
• --lk, --left-keep-field-names {a,b,c} — if you supply this, it means keep only the specified field names from the left file. It automatically includes the join-field name(s). Super helpful when you only want a limited subset of info from the left file. Tip: you can do --lk "" — that means the left file becomes solely a row-selector for the input files. Iconic behaviour honestly.
• --lp {text} — extra prefix for non-join output field names coming from the left file. Applies to paired and unpaired output records.
• --rp {text} — extra prefix for non-join output field names coming from the right file(s). Also applies to paired and unpaired output records.
• --np — do not emit paired records.
• --ul — emit unpaired records from the left file.
• --ur — emit unpaired records from the right file(s).
• --ignore-empty — treat records with empty-string values in any join-field as if that join-field were absent, on both the left and right files. Such records are never paired — not even with one another — and are treated as unpaired, subject to --np/--ul/--ur as usual. Read that twice, it matters 🫶
• -s, --sorted-input — require sorted input: records must be sorted lexically by their join-field names, else not all records will be paired. The only likely use case for this is with a left file which is too big to fit into system memory otherwise.
• -u — enable unsorted input. (This is the default even without -u.) In this case, the entire left file will be loaded into memory.
• --prepipe {command} — shell command to prepipe the left-file input through. As in main input options; see mlr --help for details. Heads up: if you wish to use a prepipe command for the main input as well as here, it must be specified there as well as here.
• --prepipex {command} — shell command to prepipe the left-file input through (no shell quoting). Same deal as above, as in main input options; see mlr --help for details.
• -h, --help — show this message.

File-format options default to those for the right file names on the Miller argument list, but they may be overridden for the left file as follows. Please see the main mlr --help for more information on syntax for these arguments:

• -i {format name} for the left-file format, e.g. 'csv' or 'json'. Or, flags like --icsv, --ijson, etc.: --icsv is the same as -i csv, and so on.
• --irs {record-separator character}
• --ifs {field-separator character}
• --ips {pair-separator character}
• --repifs
• --implicit-csv-header
• --implicit-tsv-header
• --no-implicit-csv-header
• --no-implicit-tsv-header

For example, if you have mlr --csv ... join -l foo ... then the left-file format will be specified CSV as well unless you override with mlr --csv ... join --ijson -l foo etc. Likewise, if you have mlr --csv --implicit-csv-header ... then the join-in file will be expected to be headerless as well unless you put --no-implicit-csv-header after join. No cap, that trips everyone up once 😭

Please use mlr --usage-separator-options for information on specifying separators.

Please see https://miller.readthedocs.io/en/latest/reference-verbs#join for more information including examples.

---

label 🏷️

Usage: mlr label [options] {new1,new2,new3,...}

Give it n comma-separated names and it renames the first n fields of each record to have the respective name. (Fields past the *n*th are left with their original names.) Particularly useful with --inidx or --implicit-csv-header, to give useful names to otherwise integer-indexed fields. Like giving your group chat a glow-up ✨

Options:

• -h, --help — show this message.

---

latin1-to-utf8 🌈

Usage: mlr latin1-to-utf8, with no options.

Recursively converts record strings from Latin-1 to UTF-8. For field-level control, please see the latin1_to_utf8 DSL function.

Options:

• -h, --help — show this message.

---

least-frequent 🥉

Usage: mlr least-frequent [options]

Shows the least frequently occurring distinct values for specified field names. The first entry is the statistical anti-mode; the remaining are runners-up. Everyone gets a shoutout, even the flops 💅

Options:

• -f {a,b,c} — one or more comma-separated field names to group by. Required flag.
• -n {count} — maximum number of results to output. Optional flag, defaulting to 10.
• -b — suppress counts; show only field values.
• -o {name} — field name for output count. Default "count".
• -h, --help — show this message.

See also mlr most-frequent.

---

merge-fields 🧬

Usage: mlr merge-fields [options]

Computes univariate statistics for each input record, accumulated across specified fields. Basically it scoops a bunch of fields into one big statistical soup 🍓

Options:

• -a {sum,count,...} — names of accumulators: one or more of the accumulators listed below.
• -f {a,b,c} — value-field names on which to compute statistics. Requires -o.
• -r {a,b,c} — regular expressions for value-field names on which to compute statistics. Requires -o.
• -c {a,b,c} — substrings for collapse mode: all fields which have the same names after removing substrings will be accumulated together. Please see examples below.
• -i — use interpolated percentiles, like R's type=7; default like type=1. Not sensical for string-valued fields. (Yeah, it says "sensical", we're keeping it.)
• -o {name} — output field basename for -f/-r.
• -k — keep the input fields which contributed to the output statistics; the default is to omit them.
• -S — no-op flag for backward compatibility with Miller 5.
• -F — no-op flag for backward compatibility with Miller 5.
• -h, --help — show this message.

Accumulators for -a:

• count — count instances of fields
• null_count — count number of empty-string/JSON-null instances per field
• distinct_count — count number of distinct values per field
• mode — find most-frequently-occurring values for fields; first-found wins tie
• antimode — find least-frequently-occurring values for fields; first-found wins tie
• sum — compute sums of specified fields
• mean — compute averages (sample means) of specified fields
• mad — compute mean absolute deviation
• var — compute sample variance of specified fields
• stddev — compute sample standard deviation of specified fields
• meaneb — estimate error bars for averages (assuming no sample autocorrelation)
• skewness — compute sample skewness of specified fields
• kurtosis — compute sample kurtosis of specified fields
• min — compute minimum values of specified fields
• max — compute maximum values of specified fields
• minlen — compute minimum string-lengths of specified fields
• maxlen — compute maximum string-lengths of specified fields

String-valued data makes sense unless arithmetic on them is required, e.g. for sum, mean, interpolated percentiles, etc. In case of mixed data, numbers are less than strings. Lowkey a whole vibe rule, remember it 🦋

Example input data: "a_in_x=1,a_out_x=2,b_in_y=4,b_out_x=8".

Example: mlr merge-fields -a sum,count -f a_in_x,a_out_x -o foo

produces "b_in_y=4,b_out_x=8,foo_sum=3,foo_count=2" since "a_in_x,a_out_x" are summed over.

Example: mlr merge-fields -a sum,count -r in_,out_ -o bar

produces "bar_sum=15,bar_count=4" since all four fields are summed over.

Example: mlr merge-fields -a sum,count -c in_,out_

produces "a_x_sum=3,a_x_count=2,b_y_sum=4,b_y_count=1,b_x_sum=8,b_x_count=1" since "a_in_x" and "a_out_x" both collapse to "a_x", "b_in_y" collapses to "b_y", and "b_out_x" collapses to "b_x". Collapse mode ate, honestly 👑

---

most-frequent 👑

Usage: mlr most-frequent [options]

Shows the most frequently occurring distinct values for specified field names. The first entry is the statistical mode; the remaining are runners-up. Main character values only 💖

Options:

• -f {a,b,c} — one or more comma-separated field names to group by. Required flag.
• -n {count} — maximum number of results to output. Optional flag, defaulting to 10.
• -b — suppress counts; show only field values.
• -o {name} — field name for output count. Default "count".
• -h, --help — show this message.

See also mlr least-frequent.

nest 🪆

Usage: mlr nest [options]

Explodes specified field values into separate fields/records, or reverses this. It’s giving “unzip the data, bestie” energy 💅

Options:

• --explode — Explode field values into separate fields/records. One of --explode or --implode is required.
• --implode — Reverse of --explode. One of --explode or --implode is required.
• --values — Operate on field values. One of --values or --pairs is required.
• --pairs — Operate on field key-value pairs. One of --values or --pairs is required.
• --across-records — Explode/implode across records. One of --across-records or --across-fields is required.
• --across-fields — Explode/implode across fields. One of --across-records or --across-fields is required.
• -f {field name} — Required field name to operate on.
• -r {field names} — Like -f but treat arguments as a regular expression. Match all field names and operate on each in record order. Example: -r '^[xy]$'.
• --nested-fs {string} — Field separator for nested values. Defaults to ";".
• --nested-ps {string} — Pair separator for nested key-value pairs. Defaults to ":".
• --evar {string} — Shorthand for --explode --values --across-records --nested-fs {string}.
• --ivar {string} — Shorthand for --implode --values --across-records --nested-fs {string}.
• -h, --help — Show this message.

Please use mlr --usage-separator-options for information on specifying separators.

Examples:

mlr nest --explode --values --across-records -f x
with input record "x=a;b;c,y=d" produces output records
  "x=a,y=d"
  "x=b,y=d"
  "x=c,y=d"
Use --implode to do the reverse.

mlr nest --explode --values --across-fields -f x
with input record "x=a;b;c,y=d" produces output records
  "x_1=a,x_2=b,x_3=c,y=d"
Use --implode to do the reverse.

mlr nest --explode --pairs --across-records -f x
with input record "x=a:1;b:2;c:3,y=d" produces output records
  "a=1,y=d"
  "b=2,y=d"
  "c=3,y=d"

mlr nest --explode --pairs --across-fields -f x
with input record "x=a:1;b:2;c:3,y=d" produces output records
  "a=1,b=2,c=3,y=d"

Notes:

• With --pairs, --implode doesn’t make sense since the original field name has been lost.
• The combination "--implode --values --across-records" is non-streaming: no output records are produced until all input records have been read. In particular, this means it won’t work in tail -f contexts. But all other flag combinations result in streaming (tail -f friendly) data processing. If input is coming from tail -f, be sure to use --records-per-batch 1.
• It’s up to you to ensure that the nested-fs is distinct from your data’s IFS: e.g. by default the former is semicolon and the latter is comma.

See also mlr reshape.

---

nothing 🌫️

Usage: mlr nothing [options]

Drops all input records. Useful for testing, or after tee/print/etc. have produced other output. It’s literally the “leave no trace” verb 🫶

Options:

• -h, --help — Show this message.

---

put ✨

Usage: mlr put [options] {DSL expression}

Lets you use a domain-specific language to programmatically alter stream records. This is main-character energy for your data 🔥

See also: https://miller.readthedocs.io/en/latest/reference-verbs

Options:

• -f {file name} — File containing a DSL expression (see examples below). If the filename is a directory, all *.mlr files in that directory are loaded.
• -e {expression} — DSL expression to evaluate. You can use this after -f to add an expression. Example use case: define functions/subroutines in a file you specify with -f, then call them with an expression you specify with -e.
• -s {name=value} — Predefines out-of-stream variable @name to have the given value. Thus mlr put -s foo=97 '$column += @foo' is like mlr put 'begin {@foo = 97} $column += @foo'. The value part is subject to type-inferencing. May be specified more than once, e.g. -s name1=value1 -s name2=value2. Note: the value may be an environment variable, e.g. -s sequence=$SEQUENCE.
• -x — Prints records for which {expression} evaluates to false, not true, i.e. invert the sense of the filter expression. Default false.
• -q — Does not include the modified record in the output stream. Useful for when all desired output is in begin and/or end blocks.
• -S — No-op in Miller 6 and above, since type-inferencing is now done by the record-readers before filter/put is executed. Supported as a no-op pass-through flag for backward compatibility.
• -F — No-op in Miller 6 and above, since type-inferencing is now done by the record-readers before filter/put is executed. Supported as a no-op pass-through flag for backward compatibility.
• -w — Print warnings about things like uninitialized variables.
• -W — Same as -w, but exit the process if there are any warnings.
• -p — Prints the expression’s AST (abstract syntax tree), which gives full transparency on the precedence and associativity rules of Miller’s grammar, to stdout.
• -d — Like -p but uses a parenthesized-expression format for the AST.
• -D — Like -d but with output all on one line.
• -E — Echo DSL expression before printing parse-tree.
• -v — Same as -E -p.
• -X — Exit after parsing but before stream-processing. Useful with -v/-d/-D, if you only want to look at parser information.
• --explain — Parse and type-check the DSL expression, report whether it is valid, and exit without reading the input stream. Exit status is 0 if the expression is valid and non-zero otherwise; combine with --errors-json for a machine-readable error.
• -h, --help — Show this message.

If you mix -e and -f then the expressions are evaluated in the order encountered. Since the expression pieces are simply concatenated, please be sure to use intervening semicolons to separate expressions.

Parser-info options are -w, -W, -p, -d, -D, -E, -v, and -X.

Examples:

mlr --from example.csv put '$qr = $quantity * $rate'

More example put expressions:

If-statements:
  'if ($flag == true) { $quantity *= 10}'
  'if ($x > 0.0) { $y=log10($x); $z=sqrt($y) } else {$y = 0.0; $z = 0.0}'
Newly created fields can be read after being written:
  '$new_field = $index**2; $qn = $quantity * $new_field'
Regex-replacement:
  '$name = sub($name, "http.*com"i, "")'
Regex-capture:
   'if ($a =~ "([a-z]+)_([0-9]+)") { $b = "left_\1"; $c = "right_\2" }'
Built-in variables:
  '$filename = FILENAME'
Aggregations (use mlr put -q):
  '@sum += $x; end {emit @sum}'
  '@sum[$shape] += $quantity; end {emit @sum, "shape"}'
  '@sum[$shape][$color] += $x; end {emit @sum, "shape", "color"}'
  '
    @min = min(@min,$x);
    @max=max(@max,$x);
    end{emitf @min, @max}
  '

See also https://miller.readthedocs.io/reference-dsl for more context.

---

rank 🏆

Usage: mlr rank [options]

For each record’s value in specified fields, computes the standard competition rank (1,2,2,4,...) of that value among all input records, optionally within groups.

E.g. with input records x=10, x=20, x=20, and x=30, emits output records x=10,x_rank=1 x=20,x_rank=2 x=20,x_rank=2 and x=30,x_rank=4.

Note: by default this is a two-pass algorithm: on the first pass it retains input records and their values; on the second pass it computes ranks and emits output records, in original input order. This means it produces no output until all input is read, but gives correct ranks regardless of input order. Use --sorted for a single-pass streaming alternative.

Options:

• -f {a,b,c} — Field name(s) to rank.
• -g {d,e,f} — Optional group-by-field name(s).
• --sorted — Promise that the input is already sorted by the field(s) being ranked (within each group, if -g is given). This computes rank in a single streaming pass and O(1) space, by comparing each record’s value only to the immediately preceding one, rather than buffering all records to compute an order-independent rank. Produces wrong output if the input is not in fact sorted.
• -h, --help — Show this message.

Examples:

mlr rank -f x data/rank-example.csv

mlr rank -f x -g g data/rank-example.csv

mlr sort -f x then rank -f x --sorted data/rank-example.csv

---

regularize 🧼

Usage: mlr regularize [options]

Outputs records sorted lexically ascending by keys. Clean, ordered, iconic ✨

Options:

• -h, --help — Show this message.

---

remove-empty-columns 🫧

Usage: mlr remove-empty-columns [options]

Omits fields which are empty on every input row. Non-streaming.

Options:

• -h, --help — Show this message.

---

rename 🏷️

Usage: mlr rename [options] {old1,new1,old2,new2,...}

Renames specified fields. Glow-up for your column names 💅

Options:

• -r — Treat old field names as regular expressions. "ab", "a.*b" will match any field name containing the substring "ab" or matching "a.*b", respectively; anchors of the form "^ab$", "^a.*b$" may be used. New field names may be plain strings, or may contain capture groups of the form "\1" through "\9". Wrapping the regex in double quotes is optional, but is required if you wish to follow it with 'i' to indicate case-insensitivity.
• -g — Do global replacement within each field name rather than first-match replacement.
• -h, --help — Show this message.

Examples:

mlr rename old_name,new_name
mlr rename old_name_1,new_name_1,old_name_2,new_name_2
mlr rename -r 'Date_[0-9]+,Date'  Rename all such fields to be "Date"
mlr rename -r '"Date_[0-9]+",Date' Same
mlr rename -r 'Date_([0-9]+).*,\1' Rename all such fields to be of the form 20151015
mlr rename -r '"name"i,Name'       Rename "name", "Name", "NAME", etc. to "Name"

🌀 reorder — bestie, we're moving fields around

Usage:

mlr reorder [options]

Moves specified names to start of record, or end of record. Like, you're the DJ and the fields are the playlist. 💅

Options:

• -e — Put specified field names at record end; default is to put them at record start.
• -f {a,b,c} — Field names to reorder.
• -r {a,b,c} — Treat field names as regular expressions. Matched fields are moved to start or end, grouped by the order the regexes are given; within each group, fields keep their record order. Example: -r '^YYY,^XXX' puts all YYY-prefixed fields first, then all XXX-prefixed fields, then the rest. Regex glow-up, no cap. ✨
• -b {x} — Put field names specified with -f before field name specified by {x}, if any. If {x} isn't present in a given record, the specified fields will not be moved.
• -a {x} — Put field names specified with -f after field name specified by {x}, if any. If {x} isn't present in a given record, the specified fields will not be moved.
• -h, --help — Show this message.

Examples (iconic behaviour):

mlr reorder -f a,b sends input record "d=4,b=2,a=1,c=3" to "a=1,b=2,d=4,c=3".

mlr reorder -e -f a,b sends input record "d=4,b=2,a=1,c=3" to "d=4,c=3,a=1,b=2".

mlr reorder -r '^YYY,^XXX' puts YYY-prefixed fields first, then XXX-prefixed fields, then rest.

---

🔁 repeat — one record, but make it a franchise

Usage:

mlr repeat [options]

Copies input records to output records multiple times. It's giving "you get a copy, and you get a copy" 🫶

Options must be exactly one of -n or -f. Pick one, bestie, not both.

Options:

• -n {repeat count} — Repeat each input record this many times.
• -f {field name} — Same as -n, but take the repeat count from the specified field name of each input record.
• -h, --help — Show this message.

Example:

echo x=0 | mlr repeat -n 4 then put '$x=urand()'

produces:

x=0.488189
x=0.484973
x=0.704983
x=0.147311

Example:

echo a=1,b=2,c=3 | mlr repeat -f b

produces:

a=1,b=2,c=3
a=1,b=2,c=3

Example:

echo a=1,b=2,c=3 | mlr repeat -f c

produces:

a=1,b=2,c=3
a=1,b=2,c=3
a=1,b=2,c=3

---

🧬 reshape — the wide↔long glow-up

Usage:

mlr reshape [options]

Wide-to-long options (the "spread it out" era):

• -i {input field names} + -o {key-field name,value-field name}
• -r {input field regex} + -o {key-field name,value-field name}

These pivot/reshape the input data such that the input fields are removed and separate records are emitted for each key/value pair.

Note: if you have multiple regexes, please specify them using multiple -r, since regexes can contain commas within them. (Commas inside regexes = chaos, we don't do that 💀)

Note: this works with tail -f and produces output records for each input record seen. If input is coming from tail -f, be sure to use --records-per-batch 1.

Long-to-wide options (the "bring it back together" era):

• -s {key-field name,value-field name} — These pivot/reshape the input data to undo the wide-to-long operation.

Note: this does not work with tail -f; it produces output records only after all input records have been read. Patience, queen. 🎀

Options:

• -i {input field names} — Input field names for wide-to-long reshape. Use with -o.
• -r {input field regex} — Input field regex for wide-to-long reshape. May be repeated. Use with -o. If you have multiple regexes, please specify them using multiple -r, since regexes can contain commas within them.
• -o {key-field name,value-field name} — Output key-field and value-field names for wide-to-long reshape. Requires -i or -r.
• -s {key-field name,value-field name} — Key-field and value-field names for long-to-wide reshape.
• -S — Sort the newly-created wide-field keys for long-to-wide reshape (-s). Lexically ascending by default; use -n for natural ordering. Without -S, new fields appear in order of first occurrence. Other (non-pivoted) fields keep their input order.
• -n — With -S, sort the new wide-field keys naturally (e.g. 2 before 12) rather than lexically.
• -h, --help — Show this message.

Examples:

Input file "wide.txt":

  time       X           Y
  2009-01-01 0.65473572  2.4520609
  2009-01-02 -0.89248112 0.2154713
  2009-01-03 0.98012375  1.3179287

mlr --pprint reshape -i X,Y -o item,value wide.txt
  time       item value
  2009-01-01 X    0.65473572
  2009-01-01 Y    2.4520609
  2009-01-02 X    -0.89248112
  2009-01-02 Y    0.2154713
  2009-01-03 X    0.98012375
  2009-01-03 Y    1.3179287

mlr --pprint reshape -r '[A-Z]' -o item,value wide.txt
  time       item value
  2009-01-01 X    0.65473572
  2009-01-01 Y    2.4520609
  2009-01-02 X    -0.89248112
  2009-01-02 Y    0.2154713
  2009-01-03 X    0.98012375
  2009-01-03 Y    1.3179287

Input file "long.txt":

  time       item value
  2009-01-01 X    0.65473572
  2009-01-01 Y    2.4520609
  2009-01-02 X    -0.89248112
  2009-01-02 Y    0.2154713
  2009-01-03 X    0.98012375
  2009-01-03 Y    1.3179287

mlr --pprint reshape -s item,value long.txt
  time       X           Y
  2009-01-01 0.65473572  2.4520609
  2009-01-02 -0.89248112 0.2154713
  2009-01-03 0.98012375  1.3179287

See also mlr nest. 🪺

---

🎲 sample — reservoir sampling, but make it fair

Usage:

mlr sample [options]

Reservoir sampling (subsampling without replacement), optionally by category. See also mlr bootstrap and mlr shuffle. It's giving "everybody gets an equal chance" energy. ✨

Options:

• -g {a,b,c} — Optional: group-by-field names for samples, e.g. a,b,c.
• -k {k} — Required: number of records to output in total, or by group if using -g.
• -h, --help — Show this message.

---

📅 sec2gmtdate — seconds → pretty date, keystroke-saver edition

Usage:

mlr sec2gmtdate {comma-separated list of field names}

Replaces a numeric field representing seconds since the epoch with the corresponding GMT year-month-day timestamp; leaves non-numbers as-is. This is nothing more than a keystroke-saver for the sec2gmtdate function:

mlr sec2gmtdate time1,time2

is the same as

mlr put '$time1=sec2gmtdate($time1);$time2=sec2gmtdate($time2)'

Options:

• -h, --help — Show this message.

---

⏰ sec2gmt — seconds → full GMT timestamp

Usage:

mlr sec2gmt [options] {comma-separated list of field names}

Replaces a numeric field representing seconds since the epoch with the corresponding GMT timestamp; leaves non-numbers as-is. This is nothing more than a keystroke-saver for the sec2gmt function:

mlr sec2gmt time1,time2

is the same as

mlr put '$time1 = sec2gmt($time1); $time2 = sec2gmt($time2)'

Options:

• -1 — Format seconds with 1 decimal place.
• -2 — Format seconds with 2 decimal places.
• -3 — Format seconds with 3 decimal places.
• -4 — Format seconds with 4 decimal places.
• -5 — Format seconds with 5 decimal places.
• -6 — Format seconds with 6 decimal places.
• -7 — Format seconds with 7 decimal places.
• -8 — Format seconds with 8 decimal places.
• -9 — Format seconds with 9 decimal places.
• --millis — Input numbers are treated as milliseconds since the epoch.
• --micros — Input numbers are treated as microseconds since the epoch.
• --nanos — Input numbers are treated as nanoseconds since the epoch.
• -h, --help — Show this message.

---

🔢 seqgen — the counter machine, input not invited

Usage:

mlr seqgen [options]

Produces a sequence of counters. Discards the input record stream. Produces output as specified by the options. Like, your input? Doesn't matter. Bye. 💀

Options:

• -f {name} — Field name for counters. Default "i".
• --start {value} — Inclusive start value. Default 1.
• --step {value} — Step value. Default 1. May be negative but not zero (unless start == stop).
• --stop {value} — Inclusive stop value. Default 100.
• -h, --help — Show this message.

Start, stop, and/or step may be floating-point. Output is integer if start, stop, and step are all integers. Step may be negative. It may not be zero unless start == stop.

---

🃏 shuffle — randomize the whole vibe

Usage:

mlr shuffle [options]

Outputs records randomly permuted. No output records are produced until all input records are read. See also mlr bootstrap and mlr sample.

Options:

• -h, --help — Show this message.

---

🧹 skip-trivial-records — goodbye, empty nonsense

Usage:

mlr skip-trivial-records [options]

Passes through all records except those with zero fields, or those for which all fields have empty value. If your record is giving nothing, it's not invited to the function. 🙈

Options:

• -h, --help — Show this message.

---

🔤 sort — the main character of ordering

Usage:

mlr sort {flags}

Sorts records primarily by the first specified field, secondarily by the second field, and so on. (Any records not having all specified sort keys will appear at the end of the output, in the order they were encountered, regardless of the specified sort order.) The sort is stable: records that compare equal will sort in the order they were encountered in the input record stream. Stable sort rights! 👑

Options:

• -f {a,b,c} — Lexical ascending sort on the specified field names.
• -r {a,b,c} — Lexical descending sort on the specified field names.
• -c {a,b,c} — Case-folded lexical ascending sort on the specified field names.
• -cr {a,b,c} — Case-folded lexical descending sort on the specified field names.
• -n {a,b,c} — Numerical ascending sort on the specified field names; nulls sort last.
• -nf {a,b,c} — Same as -n.
• -nr {a,b,c} — Numerical descending sort on the specified field names; nulls sort first.
• -t {a,b,c} — Natural ascending sort on the specified field names.
• -b — Move sort fields to start of record, as in reorder -b.
• -tr, -rt {a,b,c} — Natural descending sort on the specified field names.
• -h, --help — Show this message.

Example:

mlr sort -f a,b -nr x,y,z

which is the same as:

mlr sort -f a -f b -nr x -nr y -nr z

Same energy, less typing. 💅

---

🗂️ sort-within-records — sorting inside the record itself

Usage:

mlr sort-within-records [options]

Outputs records sorted lexically ascending by keys. It's the "organize your room" of verbs. 🌸

Options:

• -f {names} — Sort only these keys; others preserve record order.
• -r {regex} — Sort only keys matching this regex; others preserve record order. Example: -r '^[xy]' sorts keys starting with x or y. With no regex argument, -r recursively sorts subobjects/submaps (e.g. for JSON input), or combines with -f to treat names as regexes.
• -n — Sort field names naturally (e.g. 2 before 12). Combines with -f/-r.
• -h, --help — Show this message.

✨ sparkline ✨

Usage: mlr sparkline [options]

okay so this one is so cute — it takes your numeric field(s) and squishes them, in input order, down to one tiny compact Unicode sparkline 🎀 like literally one block character per record, so you can see the trend with your eyeballs instead of squinting at a million numbers. It emits one output record per field, and heads up bestie: it holds all records in memory before producing any output. So don't point it at an infinite stream, mkay? 😭

The options, slay:

• -f {a,b,c} — the field names you want to sparkline. that's the vibe.
• -h, --help — show this message. always here for you 🫶

---

🌸 sparsify 🌸

Usage: mlr sparsify [options]

this one unsets fields where the key is the empty string (or, if you want, some other value you specify). Lowkey only makes sense when your output format is not CSV or TSV — like, CSV can't really do "absent", y'know? 💅

• -s {filler string} — what values to remove. Defaults to the empty string.
• -f {a,b,c} — specify field names to be operated on; any other fields won't be modified. The default is to modify all fields.
• -h, --help — show this message.

Example: if input is a=1,b=,c=3 then output is a=1,c=3. literally it just ghosted that empty field 👻

---

🦋 split 🦋

Usage: mlr split [options] {filename}

bestie this is the file-splitting icon 👑 it takes your stream and chops it into multiple output files. options:

• -n {n} — cap output file sizes at N records.
• -m {m} — produce M files, round-robining records among them.
• -g {a,b,c} — write separate files with records having distinct values for the specified field names.
• --prefix {p} — output filename prefix. Default "split".
• --suffix {s} — output filename suffix. Default is from the output format, e.g. "csv".
• --folder {f} — output directory. Default is current directory.
• -a — append to existing files rather than overwriting. respectful 🫶
• -v — send records downstream as well as splitting to files.
• -e — do NOT URL-escape names of output files.
• -j {J} — string used to join filename parts. Default "_".
• -h, --help — show this message.

⚠️ Exactly one of -m, -n, or -g must be supplied. not two, not zero. one. periodt.

any of the output-format command-line flags work too (see mlr -h). for example, using

mlr --icsv --from myfile.csv split --ojson -n 1000

the input is CSV, but the output files are JSON. iconic behavior fr fr ✨

Examples: suppose myfile.csv has 1,000,000 records. 😳

100 output files, 10,000 records each. First 10,000 records in split_1.csv, next in split_2.csv, etc.

mlr --csv --from myfile.csv split -n 10000

10 output files, 100,000 records each. Records 1,11,21,etc in split_1.csv, records 2,12,22, etc in split_2.csv, etc.

mlr --csv --from myfile.csv split -m 10

Same, but with JSON output.

mlr --csv --from myfile.csv split -m 10 -o json

Same but instead of split_1.csv, split_2.csv, etc. there are test_1.dat, test_2.dat, etc.

mlr --csv --from myfile.csv split -m 10 --prefix test --suffix dat

Same, but written to the /tmp/ directory.

mlr --csv --from myfile.csv split -m 10 --prefix /tmp/test --suffix dat

Or using --folder:

mlr --csv --from myfile.csv split -m 10 --folder /tmp --prefix test --suffix dat

If the shape field has values triangle and square, then there will be split_triangle.csv and split_square.csv.

mlr --csv --from myfile.csv split -g shape

If the color field has values yellow and green, and the shape field has values triangle and square, then there will be split_yellow_triangle.csv, split_yellow_square.csv, etc.

mlr --csv --from myfile.csv split -g color,shape

See also the "tee" DSL function which lets you do more ad-hoc customization. 💖

---

🎀 ssub 🎀

Usage: mlr ssub [options]

replaces old string with new string in specified field(s), without regex support for the old string — like the ssub DSL function. it's the chill, no-regex sibling of gsub/sub 🧚

Both the search and replacement strings support C-style backslash escapes such as \n, \t, and \x1f. Write \\ to get a literal backslash. don't forget the double-slash bestie 😭

See also the gsub and sub verbs.

• -f {a,b,c} — field names to apply substitution to.
• -r {regex} — regular expression for field names to apply substitution to.
• -a — apply substitution to all fields.
• -h, --help — show this message.

---

🔥 stats1 🔥

Usage: mlr stats1 [options]

the univariate-statistics main character 💅 computes univariate statistics for one or more given fields, accumulated across the input record stream. numbers go in, stats come out.

• -a {sum,count,...} — names of accumulators: one or more of the listed values (full list below, keep scrolling 👇). Also accepts median (same as p50) and percentiles p{n} for n in 0..100, e.g. p10 p25.2 p50 p98 p100.
• -f {a,b,c} — value-field names on which to compute statistics.
• --fr {regex} — regex for value-field names on which to compute statistics (compute statistics on values in all field names matching the regex).
• --fx {regex} — inverted regex for value-field names on which to compute statistics (compute statistics on values in all field names not matching the regex).
• -g {d,e,f} — optional group-by-field names.
• --gr {regex} — regex for optional group-by-field names (group by values in field names matching the regex).
• --gx {regex} — inverted regex for optional group-by-field names (group by values in field names not matching the regex).
• --grfx {regex} — shorthand for --gr {regex} --fx {that same regex}.
• -i — use interpolated percentiles, like R's type=7; default like type=1. Not sensical for string-valued fields.
• -s — print iterative stats. Useful in tail -f contexts, in which case please avoid pprint-format output since end of input stream will never be seen. Likewise, if input is coming from tail -f be sure to use --records-per-batch 1. 🫶
• -w {n} — sliding-window mode: compute statistics over a trailing window of up to n records (including the current one), rather than over the whole record stream. Windows are kept per group when -g is used. One output record is emitted per input record, with the windowed statistics appended to it. Not compatible with -s. 🪟
• -S — no-op flag for backward compatibility with Miller 5.
• -F — no-op flag for backward compatibility with Miller 5.
• -h, --help — show this message.

Names of accumulators for -a, one or more of:

• median — this is the same as p50
• p10 p25.2 p50 p98 p100 etc. — percentiles, you know the drill
• count — count instances of fields
• null_count — count number of empty-string/JSON-null instances per field
• distinct_count — count number of distinct values per field
• mode — find most-frequently-occurring values for fields; first-found wins tie
• antimode — find least-frequently-occurring values for fields; first-found wins tie
• sum — compute sums of specified fields
• mean — compute averages (sample means) of specified fields
• mad — compute mean absolute deviation
• var — compute sample variance of specified fields
• stddev — compute sample standard deviation of specified fields
• meaneb — estimate error bars for averages (assuming no sample autocorrelation)
• skewness — compute sample skewness of specified fields
• kurtosis — compute sample kurtosis of specified fields
• min — compute minimum values of specified fields
• max — compute maximum values of specified fields
• minlen — compute minimum string-lengths of specified fields
• maxlen — compute maximum string-lengths of specified fields

Examples:

mlr stats1 -a min,p10,p50,p90,max -f value -g size,shape

mlr stats1 -a count,mode -f size

mlr stats1 -a count,mode -f size -g shape

mlr stats1 -a mean,min,max -f quantity -g name -w 7

that last one emits one output record per input record, with sliding-window statistics over the last up-to-7 records for each name. 🪟✨

mlr stats1 -a count,mode --fr '^[a-h].*$' --gr '^k.*$'

and that one computes count and mode statistics on all field names beginning with a through h, grouped by all field names starting with k. regex girlies rise 🧚‍♀️

Notes (read these, they're giving important ⚠️):

• p50 and median are synonymous.
• min and max output the same results as p0 and p100, respectively, but use less memory. efficient king 👑
• String-valued data make sense unless arithmetic on them is required, e.g. for sum, mean, interpolated percentiles, etc. In case of mixed data, numbers are less than strings.
• count and mode allow text input; the rest require numeric input. In particular, 1 and 1.0 are distinct text for count and mode. 💀
• When there are mode ties, the first-encountered datum wins.

---

💜 stats2 💜

Usage: mlr stats2 [options]

the bivariate one 👯‍♀️ computes bivariate statistics for one or more given field-name pairs, accumulated across the input record stream. two fields, one stat, immaculate vibes.

• -a {linreg-ols,corr,...} — names of accumulators: one or more of the listed values (list below 💖).
• -f {a,b,c,d} — value-field name-pairs on which to compute statistics. There must be an even number of names. count them, bestie 🔢
• -g {e,f,g} — optional group-by-field names.
• -v — print additional output for linreg-pca.
• -s — print iterative stats. Useful in tail -f contexts, in which case please avoid pprint-format output since end of input stream will never be seen. Likewise, if input is coming from tail -f, be sure to use --records-per-batch 1.
• --fit — rather than printing regression parameters, applies them to the input data to compute new fit fields. All input records are held in memory until end of input stream. Has effect only for linreg-ols, linreg-pca, and logireg. 🧠 (memory girlie, be aware)
• -S — no-op flag for backward compatibility with Miller 5.
• -F — no-op flag for backward compatibility with Miller 5.
• -h, --help — show this message.

Names of accumulators for -a, one or more of:

• linreg-ols — linear regression using ordinary least squares
• linreg-pca — linear regression using principal component analysis
• r2 — quality metric for linreg-ols (linreg-pca emits its own)
• logireg — logistic regression
• corr — sample correlation
• cov — sample covariance
• covx — sample-covariance matrix

⚠️ Only one of -s or --fit may be used. pick a lane 💅

Examples:

mlr stats2 -a linreg-pca -f x,y

mlr stats2 -a linreg-ols,r2 -f x,y -g size,shape

mlr stats2 -a corr -f x,y

slay, go forth and regress ✨

step 💅

Usage: mlr step [options]

okay so mlr step computes values dependent on earlier/later records, optionally grouped by category. it’s giving math but make it streaming ✨

Options

• -a {delta,rsum,...} — names of steppers: comma-separated, one or more of the listed values.
• -f {a,b,c} — value-field names on which to compute statistics.
• -g {d,e,f} — optional group-by-field names.
• -F — computes integerable things (e.g. counter) in floating point. As of Miller 6 this happens automatically, but the flag is accepted as a no-op for backward compatibility with Miller 5 and below.
• -d {x,y,z} — weights for EWMA. 1 means current sample gets all weight (no smoothing), near under 1 is light smoothing, near over 0 is heavy smoothing. Multiple weights may be specified, e.g. "mlr step -a ewma -f sys_load -d 0.01,0.1,0.9". Default if omitted is "-d 0.5".
• -o {a,b,c} — custom suffixes for EWMA output fields. If omitted, these default to the -d values. If supplied, the number of -o values must be the same as the number of -d values.
• -h, --help — show this message.

Stepper names for -a 🎚️

comma-separated, one or more of:

• counter — count instances of field(s) between successive records.
• delta — compute differences in field(s) between successive records. Use delta or equivalently delta_1 for the previous record, or delta_{n} for n records back.
• ewma — exponentially weighted moving average over successive records.
• from-first — compute differences in field(s) from first record.
• ratio — compute ratios in field(s) between successive records. Use ratio or equivalently ratio_1 for the previous record, or ratio_{n} for n records back.
• rprod — compute running products of field(s) between successive records.
• rsum — compute running sums of field(s) between successive records.
• shift — alias for shift_lag. Use shift or equivalently shift_1 for the previous record, or shift_{n} for n records back.
• shift_lag — include value(s) in field(s) from the previous record, if any. Use shift_lag or equivalently shift_lag_1 for the previous record, or shift_lag_{n} for n records back.
• shift_lead — include value(s) in field(s) from the next record, if any. Use shift_lead or equivalently shift_lead_1 for the next record, or shift_lead_{n} for n records forward.
• slwin — sliding-window averages over m records back and n forward. E.g. slwin_7_2 for 7 back and 2 forward.

Examples 🌸

mlr step -a rsum -f request_size
mlr step -a delta -f request_size -g hostname
mlr step -a ewma -d 0.1,0.9 -f x,y
mlr step -a ewma -d 0.1,0.9 -o smooth,rough -f x,y
mlr step -a ewma -d 0.1,0.9 -o smooth,rough -f x,y -g group_name
mlr step -a slwin_9_0,slwin_0_9 -f x
mlr step -a shift_lag_12 -f sales

The shift, shift_lag, shift_lead, delta, and ratio steppers accept an optional trailing count: shift_lag_{n} refers n records back, and shift_lead_{n} refers n records forward. The plain forms are equivalent to a count of 1: e.g. shift_lag is the same as shift_lag_1.

Please see https://miller.readthedocs.io/en/latest/reference-verbs.html#filter or https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average for more information on EWMA.

sub 💅

Usage: mlr sub [options]

Replaces old string with new string in specified field(s), with regex support for the old string and not handling multiple matches, like the sub DSL function. The replacement string supports C-style backslash escapes such as \n, \t, and \x1f. Write \\ to get a literal backslash. See also the gsub and ssub verbs.

Options

• -f {a,b,c} — field names to apply substitution to.
• -r {regex} — regular expression for field names to apply substitution to.
• -a — apply substitution to all fields.
• -h, --help — show this message.

summary 🎀

Usage: mlr summary [options]

Show summary statistics about the input data.

All summarizers

• field_type — string, int, etc. -- if a column has mixed types, all encountered types are printed (see notes below).
• count — +1 for every instance of the field across all records in the input record stream.
• null_count — count of field values either empty string or JSON null.
• distinct_count — count of distinct values for the field.
• mode — most-frequently-occurring value for the field.
• sum — sum of field values.
• mean — mean of the field values.
• stddev — standard deviation of the field values.
• var — variance of the field values.
• skewness — skewness of the field values.
• minlen — length of shortest string representation for the field.
• maxlen — length of longest string representation for the field.
• min — minimum field value.
• p25 — first-quartile field value.
• median — median field value.
• p75 — third-quartile field value.
• max — maximum field value.
• iqr — interquartile range: p75 - p25.
• lof — lower outer fence: p25 - 3.0 * iqr.
• lif — lower inner fence: p25 - 1.5 * iqr.
• uif — upper inner fence: p75 + 1.5 * iqr.
• uof — upper outer fence: p75 + 3.0 * iqr.

Default summarizers

field_type count mean min max null_count distinct_count

Notes

• min, p25, median, p75, and max work for strings as well as numbers.
• Distinct-counts are computed on string representations -- so 4.1 and 4.10 are counted as distinct here.
• If the mode is not unique in the input data, the first-encountered value is reported as the mode.
• A field_type of "int-string", "empty-string", etc. means the column contains values of mixed types -- all types encountered are printed, hyphen-joined, in the order first encountered.

Options

• -a {mean,sum,etc.} — use only the specified summarizers.
• -x {mean,sum,etc.} — use all summarizers except the specified ones.
• --all — use all available summarizers.
• --transpose — show output with field names as column names.
• -h, --help — show this message.

surv 🦋

Usage: mlr surv -d {duration-field} -s {status-field}

Estimate Kaplan-Meier survival curve (right-censored).

Options

• -d {field} — name of duration field (time-to-event or censoring).
• -s {field} — name of status field (0=censored, 1=event).
• -h, --help — show this message.

tac 💜

Usage: mlr tac [options]

Prints records in reverse order from the order in which they were encountered.

Options

• -h, --help — show this message.

tail 🧚

Usage: mlr tail [options]

Passes through the last n records, optionally by category.

Options

• -g {a,b,c} — optional group-by-field names for tail counts, e.g. a,b,c.
• -n {n} — tail-count to print. Default 10. A leading + means start at the nth record rather than print the last n: e.g. -n +3 passes through all but the first 2 records, optionally by category.
• -h, --help — show this message.

tee 🌈

Usage: mlr tee [options] {filename}

Options

• -a — append to existing file, if any, rather than overwriting.
• -p — treat filename as a pipe-to command.
• -h, --help — show this message.

Any of the output-format command-line flags (see mlr -h). Example: using

mlr --icsv --opprint put '...' then tee --ojson ./mytap.dat then stats1 ...

the input is CSV, the output is pretty-print tabular, but the tee-file output is written in JSON format.

template ✨

Usage: mlr template [options]

Places input-record fields in the order specified by list of column names. If the input record is missing a specified field, it will be filled with the fill-with. If the input record possesses an unspecified field, it will be discarded.

Options

• -f {a,b,c} — comma-separated field names for template, e.g. a,b,c.
• -t {filename} — CSV file whose header line will be used for template.
• --fill-with {filler string} — what to fill absent fields with. Defaults to the empty string.
• -h, --help — show this message.

Example

• Specified fields are a,b,c.
• Input record is c=3,a=1,f=6.
• Output record is a=1,b=,c=3.

top 👑

Usage: mlr top [options]

Options

• -f {a,b,c} — value-field names for top counts.
• -g {d,e,f} — optional group-by-field names for top counts.
• -n {count} — how many records to print per category; default 1.
• -a — print all fields for top-value records; default is to print only value and group-by fields. Requires a single value-field name only.
• --max — print top largest values. This is the default.
• --min — print top smallest values; default is top largest values.
• -F — keep top values as floats even if they look like integers (ignored in Miller 6, kept for backward compatibility).
• -o {name} — field name for output indices. Default "top_idx". Ignored if -a is used.
• -h, --help — show this message.

Prints the n records with smallest/largest values at specified fields, optionally by category. If -a is given, then the top records are emitted with the same fields as they appeared in the input. Without -a, only fields from -f, fields from -g, and the top-index field are emitted. For more information please see https://miller.readthedocs.io/en/latest/reference-verbs#top.

utf8-to-latin1 🧚

Usage: mlr utf8-to-latin1, with no options.

Recursively converts record strings from UTF-8 to Latin-1. For field-level control, please see the utf8_to_latin1 DSL function.

Options

• -h, --help — show this message.

unflatten 💖

Usage: mlr unflatten [options]

Reverses flatten. Example: field with name 'a.b.c' and value 4 becomes name 'a' and value '{"b": { "c": 4 }}'.

Options

• -f {a,b,c} — comma-separated list of field names to unflatten (default all).
• -s {string} — separator, defaulting to mlr --flatsep value.
• -h, --help — show this message.

uniq 🫶

Usage: mlr uniq [options]

Prints distinct values for specified field names. With -c, same as count-distinct. For uniq, -f is a synonym for -g. Output fields are written in the order in which they are named with -g or -f, not in the order in which they appear in the input records.

To deduplicate records by one or more fields while keeping all other fields, use head: e.g. "mlr head -n 1 -g hash" keeps the first record for each distinct value of the hash field, with all fields intact.

Still in uniq's option list, bestie — yes, we’re finishing the fit check before moving on:

• -g {d,e,f} — group-by field names for uniq counts.
• -f {d,e,f} — synonym for -g. Same energy, different letter.
• -x {a,b,c} — field names to exclude for uniq; use each record's other fields instead.
• -c — show repeat counts in addition to unique values.
• -n — show only the number of distinct values.
• -o {name} — field name for output count. Default "count".
• -a — output each unique record only once. Incompatible with -g. With -c, produces unique records, with repeat counts for each. With -n, produces only one record which is the unique-record count. With neither -c nor -n, produces unique records.
• -h|--help — show this message.

---

unspace

Usage: mlr unspace [options]

Replaces spaces in record keys and/or values with _. This is helpful for PPRINT output. Cute output needs cute field names, obviously.

Options:

• -f {x} — replace spaces with specified filler character. Default "_".
• -k — unspace only keys, not keys and values.
• -v — unspace only values, not keys and values.
• -h|--help — show this message.

---

unsparsify

Usage: mlr unsparsify [options]

Prints records with the union of field names over all input records. For field names absent in a given record but present in others, fills in a value. This verb retains all input before producing any output.

Options:

• --fill-with {filler string} — what to fill absent fields with. Defaults to the empty string.
• -f {a,b,c} — specify field names to be operated on; others are not modified and operation is streaming.
• -h|--help — show this message.

Example: if the input is two records, one being 'a=1,b=2' and the other being 'b=3,c=4', then the output is the two records 'a=1,b=2,c=' and ’a=,b=3,c=4'.

---

🧮 FUNCTIONS FOR FILTER/PUT 🧮

abs

class=math #args=1 — absolute value. The glow-up that makes negatives positive.

acos

class=math #args=1 — inverse trigonometric cosine. Math girlie behavior.

acosh

class=math #args=1 — inverse hyperbolic cosine.

antimode

class=stats #args=1 — returns the least frequently occurring value in an array or map. Returns error for non-array/non-map types. Values are stringified for comparison, so for example string "1" and integer 1 are not distinct. In cases of ties, first-found wins.

Examples:

• antimode([3,3,4,4,4]) is 3
• antimode([3,3,4,4]) is 3

any

class=higher-order-functions #args=2 — given a map or array as first argument and a function as second argument, yields a boolean true if the argument function returns true for any array/map element, false otherwise. For arrays, the function should take one argument, for array element; for maps, it should take two, for map-element key and value. In either case it should return a boolean.

Examples:

Array example: any([10,20,30], func(e) {return $index == e})
Map example: any({"a": "foo", "b": "bar"}, func(k,v) {return $[k] == v})

append

class=collections #args=2 — appends second argument to end of first argument, which must be an array.

apply

class=higher-order-functions #args=2 — given a map or array as first argument and a function as second argument, applies the function to each element of the array/map. For arrays, the function should take one argument, for array element; it should return a new element. For maps, it should take two arguments, for map-element key and value; it should return a new key-value pair (i.e. a single-entry map).

Examples:

Array example: apply([1,2,3,4,5], func(e) {return e ** 3}) returns [1, 8, 27, 64, 125].
Map example: apply({"a":1, "b":3, "c":5}, func(k,v) {return {toupper(k): v ** 2}}) returns {"A": 1, "B":9, "C": 25}",

arrayify

class=collections #args=1 — walks through a nested map/array, converting any map with consecutive keys "1", "2", ... into an array. Useful to wrap the output of unflatten.

asin

class=math #args=1 — inverse trigonometric sine.

asinh

class=math #args=1 — inverse hyperbolic sine.

asserting_absent

class=typing #args=1 — aborts with an error if is_absent on the argument returns false, else returns its argument.

asserting_array

class=typing #args=1 — aborts with an error if is_array on the argument returns false, else returns its argument.

asserting_bool

class=typing #args=1 — aborts with an error if is_bool on the argument returns false, else returns its argument.

asserting_boolean

class=typing #args=1 — aborts with an error if is_boolean on the argument returns false, else returns its argument.

asserting_bytes

class=typing #args=1 — aborts with an error if is_bytes on the argument returns false, else returns its argument.

asserting_empty

class=typing #args=1 — aborts with an error if is_empty on the argument returns false, else returns its argument.

asserting_empty_map

class=typing #args=1 — aborts with an error if is_empty_map on the argument returns false, else returns its argument.

asserting_error

class=typing #args=1 — aborts with an error if is_error on the argument returns false, else returns its argument.

asserting_float

class=typing #args=1 — aborts with an error if is_float on the argument returns false, else returns its argument.

asserting_int

class=typing #args=1 — aborts with an error if is_int on the argument returns false, else returns its argument.

asserting_map

class=typing #args=1 — aborts with an error if is_map on the argument returns false, else returns its argument.

asserting_nonempty_map

class=typing #args=1 — aborts with an error if is_nonempty_map on the argument returns false, else returns its argument.

asserting_not_array

class=typing #args=1 — aborts with an error if is_not_array on the argument returns false, else returns its argument.

asserting_not_empty

class=typing #args=1 — aborts with an error if is_not_empty on the argument returns false, else returns its argument.

asserting_not_map

class=typing #args=1 — aborts with an error if is_not_map on the argument returns false, else returns its argument.

asserting_not_null

class=typing #args=1 — aborts with an error if is_not_null on the argument returns false, else returns its argument.

asserting_null

class=typing #args=1 — aborts with an error if is_null on the argument returns false, else returns its argument.

asserting_numeric

class=typing #args=1 — aborts with an error if is_numeric on the argument returns false, else returns its argument.

asserting_present

class=typing #args=1 — aborts with an error if is_present on the argument returns false, else returns its argument.

asserting_string

class=typing #args=1 — aborts with an error if is_string on the argument returns false, else returns its argument.

atan

class=math #args=1 — one-argument arctangent.

atan2

class=math #args=2 — two-argument arctangent.

atanh

class=math #args=1 — inverse hyperbolic tangent.

base64_decode

class=string #args=1 — decodes a base64-encoded string to a bytes value; use string() to interpret the result as UTF-8 text. Returns error if the input is not valid base64.

Examples:

base64_decode("aGVsbG8=") gives the bytes 68656c6c6f
string(base64_decode("aGVsbG8=")) gives "hello"

base64_encode

class=string #args=1 — encodes a string or bytes value using standard base64 encoding.

Examples:

base64_encode("hello") gives "aGVsbG8="
base64_encode(b"\xff") gives "/w=="

bitcount

class=arithmetic #args=1 — count of 1-bits.

boolean

class=conversion #args=1 — convert int/float/bool/string to boolean.

bytes

class=conversion #args=1 — convert string to bytes; bytes values pass through as-is. See string() for the reverse.

Examples:

bytes("hi") gives the bytes 6869
string(bytes("hi")) gives "hi"

capitalize

class=string #args=1 — convert string's first character to uppercase.

cbrt

class=math #args=1 — cube root.

ceil

class=math #args=1 — ceiling: nearest integer at or above.

clean_whitespace

class=string #args=1 — same as collapse_whitespace and strip, followed by type inference.

collapse_whitespace

class=string #args=1 — strip repeated whitespace from string.

concat

class=collections #args=variadic — returns the array concatenation of the arguments. Non-array arguments are treated as single-element arrays.

Examples:

concat(1,2,3) is [1,2,3]
concat([1,2],3) is [1,2,3]
concat([1,2],[3]) is [1,2,3]

contains

class=string #args=2 — returns true if the first argument contains the second as a substring. This is like saying index(arg1, arg2) >= 0 but with less keystroking. Stringifies non-string scalar inputs; raises an error if either argument is an array or map. To test for array membership, use any, e.g. any([1,2,3], func(e) {return e == $foo}).

Examples:

contains("abcde", "e") gives true
contains("abcde", "x") gives false
contains(12345, 34) gives true
contains("forêt", "ê") gives true
contains([1,2,3], 2) gives (error)

cos

class=math #args=1 — trigonometric cosine.

cosh

class=math #args=1 — hyperbolic cosine.

count

class=stats #args=1 — returns the length of an array or map. Returns error for non-array/non-map types.

Examples:

• count([7,8,9]) is 3
• count({"a":7,"b":8,"c":9}) is 3

datediff

class=time #args=3 — computes the difference between two dates, like the spreadsheet DATEDIF function. The first two arguments are int/float seconds since the epoch (as from strptime); the third is one of "y", "m", or "d" for complete years, complete months, or days between the two dates, or "ym", "yd", or "md" for months ignoring years, days ignoring years, or days ignoring months and years, respectively (case-insensitive). Differences are computed on calendar dates in GMT, ignoring any time-of-day parts. The result is negative if the first date is after the second.

Examples:

datediff(strptime("2020-01-01", "%Y-%m-%d"), strptime("2023-05-15", "%Y-%m-%d"), "y")  = 3
datediff(strptime("2020-01-01", "%Y-%m-%d"), strptime("2023-05-15", "%Y-%m-%d"), "m")  = 40
datediff(strptime("2020-01-01", "%Y-%m-%d"), strptime("2023-05-15", "%Y-%m-%d"), "d")  = 1230
datediff(strptime("2020-01-01", "%Y-%m-%d"), strptime("2023-05-15", "%Y-%m-%d"), "ym") = 4
datediff(strptime("2020-01-01", "%Y-%m-%d"), strptime("2023-05-15", "%Y-%m-%d"), "yd") = 134
datediff(strptime("2020-01-01", "%Y-%m-%d"), strptime("2023-05-15", "%Y-%m-%d"), "md") = 14

depth

class=collections #args=1 — prints maximum depth of map/array. Scalars have depth 0.

dhms2fsec

class=time #args=1 — recovers floating-point seconds as in dhms2fsec("5d18h53m20.250000s") = 500000.250000.

dhms2sec

class=time #args=1 — recovers integer seconds as in dhms2sec("5d18h53m20s") = 500000.

distinct_count

class=stats #args=1 — returns the number of distinct values in an array or map. Returns error for non-array/non-map types. Values are stringified for comparison, so for example string "1" and integer 1 are not distinct.

Examples:

distinct_count([7,8,9,7])  is 3
distinct_count([1,"1"]) is 1
distinct_count([1,1.0]) is 2

erf

class=math #args=1 — error function.

erfc

class=math #args=1 — complementary error function.

every

(class=higher-order-functions #args=2) bestie this one is a vibe check for your whole collection 💅 Give it a map or array as the first argument and a function as the second, and it yields boolean true if that function returns true for every array/map element, false otherwise. For arrays, your function should take one argument (the array element). For maps, it should take two (map-element key and value). Either way it's gotta return a boolean, no cap.

Examples:

Array example: every(["a", "b", "c"], func(e) {return $[e] >= 0})
Map example: every({"a": "foo", "b": "bar"}, func(k,v) {return $[k] == v})

---

exec

(class=system #args=variadic) ok so this is the main-character function that literally runs a whole command for you 🫶

'$output = exec( "command", ["arg1", "arg2"], {"env": ["ENV_VAR=ENV_VALUE", "ENV_VAR2=ENV_VALUE2"], "dir": "/tmp/run_command_here", "stdin_string": "this is input fed to program", "combined_output": true )'

Runs a command via executable, path, args and environment, yielding its stdout minus the final carriage return. Periodt.

Example:

exec("echo", ["I don't do", "$SHELL things"], {"env": "SHELL=sh"}) outputs "I don't do $SHELL things"

---

exp

(class=math #args=1) the exponential function, e**x. iconic math behavior ✨

expm1

(class=math #args=1) e**x - 1. that's it that's the function 🎀

---

flatten

(class=collections #args=2,3) Flattens multi-level maps down to single-level ones. Lowkey essential for nested JSON-like structures headed into non-JSON file formats like CSV. With two arguments, the first is a map (maybe $*) and the second is the flatten separator. With three arguments, the first is prefix, the second is the flatten separator, and the third is a map; so flatten($*, ".") is the same as flatten("", ".", $*). See "Flatten/unflatten: converting between JSON and tabular formats" at https://miller.readthedocs.io for more 💖

Examples:

flatten({"a":[1,2],"b":3}, ".") is {"a.1": 1, "a.2": 2, "b": 3}.
flatten("a", ".", {"b": { "c": 4 }}) is {"a.b.c" : 4}.
flatten("", ".", {"a": { "b": 3 }}) is {"a.b" : 3}.

float

(class=conversion #args=1) Convert int/float/bool/string to float. glow-up incoming 🌸

floor

(class=math #args=1) Floor: nearest integer at or below. no rounding up, we stay grounded ✨

---

fmtifnum

(class=conversion #args=2) Identical to fmtnum, except it returns the first argument as-is if the output would be an error. so chill, so forgiving 🦋

Examples:

fmtifnum(3.4, "%.6f") gives 3.400000"
fmtifnum("abc", "%.6f") gives abc"
$* = fmtifnum($*, "%.6f") formats numeric fields in the current record, leaving non-numeric ones alone

---

fmtnum

(class=conversion #args=2) Convert int/float/bool to string using a printf-style format string (https://pkg.go.dev/fmt), e.g. '$s = fmtnum($n, "%08d")' or '$t = fmtnum($n, "%.6e")'. Miller-specific extension: "%_d" and "%_f" for comma-separated thousands 👑 This function recurses on array and map values — it goes deep, iykyk.

Examples:

$y = fmtnum($x, "%.6f")
$o = fmtnum($n, "%d")
$o = fmtnum($n, "%12d")
$y = fmtnum($x, "%.6_f")
$o = fmtnum($n, "%_d")
$o = fmtnum($n, "%12_d")

---

fold

(class=higher-order-functions #args=3) Given a map or array as first argument and a function as second, this accumulates entries into a final output — think sum or product vibes. For arrays, the function should take two arguments (accumulated value and array element). For maps, it should take four arguments (accumulated key and value, plus map-element key and value), and it should return the updated accumulator as a new key-value pair, i.e. a single-entry map. The start value for the accumulator comes from the third argument. start strong bestie 💪

Examples:

Array example: fold([1,2,3,4,5], func(acc,e) {return acc + e**3}, 10000) returns 10225.
Map example: fold({"a":1, "b":3, "c": 5}, func(acck,accv,ek,ev) {return {"sum": accv+ev**2}}, {"sum":10000}) returns 10035.

---

format

(class=string #args=variadic) Using the first argument as a format string, interpolate the remaining arguments in place of each "{}" in the format string. Also supports 1-based positional placeholders like "{1}", which let arguments be reused and/or reordered. Plain "{}" placeholders consume arguments sequentially, independently of any positional placeholders. Too-few arguments are treated as the empty string, as are positional placeholders that exceed the number of arguments; too-many arguments are just discarded. And "{0}" is an error value, since positional placeholders are 1-based — no zero-indexing here 🙈

Examples:

format("{}:{}:{}", 1,2)        gives "1:2:".
format("{}:{}:{}", 1,2,3)      gives "1:2:3".
format("{}:{}:{}", 1,2,3,4)    gives "1:2:3".
format("{1}:{2}:{1}", "a","b") gives "a:b:a".
format("{2}{}:{1}{}", 3,4)     gives "43:34".

---

fsec2dhms

(class=time #args=1) Formats floating-point seconds as in fsec2dhms(500000.25) = "5d18h53m20.250000s" ⏳

fsec2hms

(class=time #args=1) Formats floating-point seconds as in fsec2hms(5000.25) = "01:23:20.250000" 🕰️

get_keys

(class=collections #args=1) Returns array of keys of map or array 🔑

get_values

(class=collections #args=1) Returns array of values of map or array — and in the array case, it returns a copy of the array 💫

gmt2localtime

(class=time #args=1,2) Convert from a GMT-time string to a local-time string. Consulting $TZ unless the second argument is supplied.

Examples:

gmt2localtime("1999-12-31T22:00:00Z") = "2000-01-01 00:00:00" with TZ="Asia/Istanbul"
gmt2localtime("1999-12-31T22:00:00Z", "Asia/Istanbul") = "2000-01-01 00:00:00"

gmt2nsec

(class=time #args=1) Parses GMT timestamp as integer nanoseconds since the epoch. tiny units, big energy ⚡

Example:

gmt2nsec("2001-02-03T04:05:06Z") = 981173106000000000

gmt2sec

(class=time #args=1) Parses GMT timestamp as integer seconds since the epoch.

Example:

gmt2sec("2001-02-03T04:05:06Z") = 981173106

---

gssub

(class=string #args=3) Like gsub but does no regexing. No characters are special. literally no special characters, everyone's equal here 🤝

Example:

gssub("ab.d.fg", ".", "X") gives "abXdXfg"

---

gsub

(class=string #args=3) '$name = gsub($name, "old", "new")' — replace all, with support for regular expressions ✨ Capture groups \1 through \9 in the new part are matched from (...) in the old part, and they must be used within the same call to gsub — they don't persist for subsequent DSL statements. So don't come back later expecting them to still be there 😭 See also =~ and regextract. See also "Regular expressions" at https://miller.readthedocs.io.

Examples:

gsub("ababab", "ab", "XY") gives "XYXYXY"
gsub("abc.def", ".", "X") gives "XXXXXXX"
gsub("abc.def", "\.", "X") gives "abcXdef"
gsub("abcdefg", "[ce]", "X") gives "abXdXfg"
gsub("prefix4529:suffix8567", "(....ix)([0-9]+)", "[\1 : \2]") gives "[prefix : 4529]:[suffix : 8567]"

---

haskey

(class=collections #args=2) True/false if map has/hasn't key, e.g. 'haskey($*, "a")' or 'haskey(mymap, mykey)', or true/false if an array index is in bounds / out of bounds. Error if the 1st argument is not a map or array. Note: -n..-1 aliases to 1..n in Miller arrays 💅

hasvalue

(class=collections #args=2) True/false if map/array has/hasn't value, e.g. 'hasvalue($*, "a")' or 'hasvalue(myarray, myvalue)' — returns true if the value is present in the collection. Error if the 1st argument is not a map or array 🔍

---

hex_decode

(class=string #args=1) Decodes a hex-encoded string to a bytes value; use string() to interpret the result as UTF-8 text. Returns error if the input is not valid hex 🧬

Examples:

hex_decode("6869") gives the bytes 6869
string(hex_decode("6869")) gives "hi"

hex_encode

(class=string #args=1) Encodes a string or bytes value as lowercase hex.

Example:

hex_encode("hi") gives "6869"

hexfmt

(class=conversion #args=1) Convert int to hex string, e.g. 255 to "0xff" 🎨

---

hms2fsec

(class=time #args=1) Recovers floating-point seconds as in hms2fsec("01:23:20.250000") = 5000.250000 ⏱️

hms2sec

(class=time #args=1) Recovers integer seconds as in hms2sec("01:23:20") = 5000 🕐

hostname

(class=system #args=0) Returns the hostname as a string. who are you? where do you live? 💻

---

index

(class=string #args=2) Returns the index (1-based) of the second argument within the first. Returns -1 if the second argument isn't a substring of the first. Stringifies non-string scalar inputs; raises an error if either argument is an array or map. Uses UTF-8 encoding to count characters, not bytes — characters, bestie, not bytes 🌈

Examples:

index("abcde", "e") gives 5
index("abcde", "x") gives -1
index(12345, 34) gives 3
index("forêt", "t") gives 5
index([1,2,3], 2) gives (error)

---

int

(class=conversion #args=1,2) Convert int/float/bool/string to int. If the second argument is omitted and the first argument is a string, the base is inferred from the first argument's prefix. If the second argument is provided and the first argument is a string, the second argument is used as the base. If the second argument is provided and the first argument is not a string, the second argument is ignored. rules are rules 💅

Examples:

int("345") gives decimal 345 (base-10/decimal input is inferred)
int("0xff") gives decimal 255 (base-16/hexadecimal input is inferred)
int("0377") gives decimal 255 (base-8/octal input is inferred)
int("0b11010011") gives decimal 211 which is hexadecimal 0xd3 (base-2/binary input is inferred)
int("0377", 10) gives decimal 377
int(345, 16) gives decimal 345
int(string(345), 16) gives decimal 837

---

invqnorm

(class=math #args=1) Inverse of normal cumulative distribution function. Note that invqorm(urand()) is normally distributed 📊✨

---

is_absent

(class=typing #args=1) False if field is present in input, true otherwise 👻

is_array

(class=typing #args=1) True if argument is an array.

is_bool

(class=typing #args=1) True if field is present with boolean value. Synonymous with is_boolean.

is_boolean

(class=typing #args=1) True if field is present with boolean value. Synonymous with is_bool.

is_bytes

(class=typing #args=1) True if field is present with bytes value.

is_empty

(class=typing #args=1) True if field is present in input with empty string value, false otherwise.

is_empty_map

(class=typing #args=1) True if argument is a map which is empty.

is_error

(class=typing #args=1) True if if argument is an error, such as taking string length of an integer 💀

is_float

(class=typing #args=1) True if field is present with value inferred to be float.

is_int

(class=typing #args=1) True if field is present with value inferred to be int.

is_map

(class=typing #args=1) True if argument is a map.

is_nan

(class=typing #args=1) True if the argument is the NaN (not-a-number) floating-point value. Note that NaN has the property that NaN != NaN, so you need 'is_nan(x)' rather than 'x == NaN' — she's mysterious, she's not equal to herself 😭

is_nonempty_map

(class=typing #args=1) True if argument is a map which is non-empty.

is_not_array

(class=typing #args=1) True if argument is not an array.

is_not_empty

(class=typing #args=1) True if field is present in input with non-empty value, false otherwise.

is_not_map

(class=typing #args=1) True if argument is not a map.

is_not_null

(class=typing #args=1) False if argument is null (empty, absent, or JSON null), true otherwise.

is_null

(class=typing #args=1) True if argument is null (empty, absent, or JSON null), false otherwise.

is_numeric

(class=typing #args=1) True if field is present with value inferred to be int or float.

is_present

(class=typing #args=1) True if field is present in input, false otherwise.

is_string

(class=typing #args=1) True if field is present with string (including empty-string) value 🎀

joink

class=conversion #args=2 — turns a map or array's keys into a string, bestie. First arg is the map/array, second arg is the separator string. That's it, that's the whole vibe ✨

Examples:

joink({"a":3,"b":4,"c":5}, ",") = "a,b,c".
joink([1,2,3], ",") = "1,2,3".

joinkv

class=conversion #args=3 — same energy but for key-value pairs, so you get both halves 🫶 First arg is the map/array, second is the pair-separator string, third is the field-separator string. Mnemonic, and this is the cute part: the "=" comes before the "," in the output and in the arguments to joinkv.

Examples:

joinkv([3,4,5], "=", ",") = "1=3,2=4,3=5"
joinkv({"a":3,"b":4,"c":5}, ":", ";") = "a:3;b:4;c:5"

joinv

class=conversion #args=2 — keys are so last season, this one stringifies the values. First arg is the map/array, second is the separator string 💖

Examples:

joinv([3,4,5], ",") = "3,4,5"
joinv({"a":3,"b":4,"c":5}, ",") = "3,4,5"

json_parse

class=collections #args=1 — takes a value that's a JSON-formatted string and converts it into a real value. Glow-up in one call 🦋

json_stringify

class=collections #args=1,2 — the reverse slay: converts a value into a JSON-formatted string. Default output is single-line. Pass the optional second boolean argument as true and you get multiline output instead, iykyk.

kurtosis

class=stats #args=1 — returns the sample kurtosis of the values in an array or map. Two caveats, no cap: returns empty string AKA void for an array/map of length less than two, and returns an error for non-array/non-map types.

Example:

kurtosis([4,5,9,10,11]) is -1.6703688

latin1_to_utf8

class=string #args=1 — tries to convert a Latin-1-encoded string into a UTF-8-encoded string. If the argument is an array or map, it recurses into it. Considerate king 👑

Examples:

$y = latin1_to_utf8($x)
$* = latin1_to_utf8($*)

leafcount

class=collections #args=1 — counts the total number of terminal values in a map/array. For a single-level map/array, same as length. For nested stuff, it actually goes all the way down 🌸

leftpad

class=string #args=3 — left-pads the first argument to at most the specified length (second argument, an integer) using the specified pad value (third argument, a string). If the first argument isn't a string, it gets stringified first, no drama.

Examples:

leftpad("abcdefg", 10 , "*") gives "***abcdefg".
leftpad("abcdefg", 10 , "XY") gives "XYabcdefg".
leftpad("1234567", 10 , "0") gives "0001234567".

length

class=collections #args=1 — counts the number of top-level entries in an array/map. And scalars? Scalars have length 1. Everyone gets a length 💅

localtime2gmt

class=time #args=1,2 — converts a local-time string into a GMT-time string. It consults $TZ unless the second argument is supplied.

Examples:

localtime2gmt("2000-01-01 00:00:00") = "1999-12-31T22:00:00Z" with TZ="Asia/Istanbul"
localtime2gmt("2000-01-01 00:00:00", "Asia/Istanbul") = "1999-12-31T22:00:00Z"

localtime2nsec

class=time #args=1,2 — parses a local timestamp as integer nanoseconds since the epoch. Consults the $TZ environment variable, unless the second argument is supplied.

Examples:

localtime2nsec("2001-02-03 04:05:06") = 981165906000000000 with TZ="Asia/Istanbul"
localtime2nsec("2001-02-03 04:05:06", "Asia/Istanbul") = 981165906000000000"

localtime2sec

class=time #args=1,2 — parses a local timestamp as integer seconds since the epoch. Consults the $TZ environment variable, unless the second argument is supplied.

Examples:

localtime2sec("2001-02-03 04:05:06") = 981165906 with TZ="Asia/Istanbul"
localtime2sec("2001-02-03 04:05:06", "Asia/Istanbul") = 981165906"

log

class=math #args=1 — natural (base-e) logarithm. Classic. Iconic. Math 💜

log10

class=math #args=1 — base-10 logarithm.

log1p

class=math #args=1 — log(1-x).

logifit

class=math #args=3 — given m and b from logistic regression, computes the fit: $yhat=logifit($x,$m,$b).

lstrip

class=string #args=1 — strips leading whitespace from a string. Clean girl aesthetic ✨

madd

class=arithmetic #args=3 — a + b mod m (integers).

mapdiff

class=collections #args=variadic — with 0 args, returns an empty map. With 1 arg, returns a copy of the arg. With 2 or more, returns a copy of arg 1 with all keys from any of the remaining argument maps removed. Subtraction but for maps 🎀

mapexcept

class=collections #args=variadic — returns a map with keys from the remaining arguments, if any, unset. Remaining arguments can be strings or arrays of string. E.g. 'mapexcept({1:2,3:4,5:6}, 1, 5, 7)' is '{3:4}' and 'mapexcept({1:2,3:4,5:6}, [1, 5, 7])' is '{3:4}'.

mapselect

class=collections #args=variadic — returns a map with only the keys from the remaining arguments set. Remaining arguments can be strings or arrays of string. E.g. 'mapselect({1:2,3:4,5:6}, 1, 5, 7)' is '{1:2,5:6}' and 'mapselect({1:2,3:4,5:6}, [1, 5, 7])' is '{1:2,5:6}'.

mapsum

class=collections #args=variadic — with 0 args, returns an empty map. With >= 1 arg, returns a map with key-value pairs from all arguments. Rightmost collisions win, e.g. 'mapsum({1:2,3:4},{1:5})' is '{1:5,3:4}'. Last one to the party gets the key 🔥

max

class=math #args=variadic — max of n numbers; null loses. And plot twist, the min and max functions also recurse into arrays and maps, so they can be used to get min/max stats on array/map values 👑

maxlen

class=stats #args=1 — returns the maximum string length of the values in an array or map. Returns empty string AKA void for an array/map of length less than two; returns an error for non-array/non-map types.

Example:

maxlen(["año", "alto"]) is 4

md5

class=hashing #args=1 — MD5 hash of a string or bytes value. Secure-ish, but mostly it's just hashing time 🦋

mean

class=stats #args=1 — returns the arithmetic mean of the values in an array or map. Returns empty string AKA void for an empty array/map; returns an error for non-array/non-map types.

Example:

mean([4,5,7,10]) is 6.5

meaneb

class=stats #args=1 — returns the error bar for the arithmetic mean of the values in an array or map, assuming the values are independent and identically distributed. Returns empty string AKA void for an array/map of length less than two; returns an error for non-array/non-map types.

Example:

meaneb([4,5,7,10]) is 1.3228756

median

class=stats #args=1,2 — returns the median of the values in an array or map. Returns empty string AKA void for an empty array/map; returns an error for non-array/non-map types. Please see the percentiles function for the tea on optional flags, and on performance for large inputs.

Examples:

median([3,4,5,6,9,10]) is 6
median([3,4,5,6,9,10],{"interpolate_linearly":true}) is 5.5
median(["abc", "def", "ghi", "ghi"]) is "ghi"

mexp

class=arithmetic #args=3 — a ** b mod m (integers).

min

class=math #args=variadic — min of n numbers; null loses. And again, the min and max functions also recurse into arrays and maps, so they can be used to get min/max stats on array/map values. Equal opportunity recursion 🌈

minlen

class=stats #args=1 — returns the minimum string length of the values in an array or map. Returns empty string AKA void for an array/map of length less than two; returns an error for non-array/non-map types.

Example:

minlen(["año", "alto"]) is 3

mmul

class=arithmetic #args=3 — a * b mod m (integers).

mode

class=stats #args=1 — returns the most frequently occurring value in an array or map. Returns an error for non-array/non-map types. Important: values are stringified for comparison, so for example string "1" and integer 1 are not distinct. In cases of ties, first-found wins. First come, first served 💅

Examples:

mode([3,3,4,4,4]) is 4
mode([3,3,4,4]) is 3

msub

class=arithmetic #args=3 — a - b mod m (integers).

next

class=system #args=0 — mlr script mode only, heads up. Reads the next record from input. Returns true if a record was read (and sets $* to it), false at end of stream (and $* becomes false). Stream's over, bestie 😭

nsec2gmt

class=time #args=1,2 — formats integer nanoseconds since epoch as a GMT timestamp. Leaves non-numbers as-is. With a second integer argument n, includes n decimal places for the seconds part.

Examples:

nsec2gmt(1234567890000000000)    = "2009-02-13T23:31:30Z"
nsec2gmt(1234567890123456789)    = "2009-02-13T23:31:30Z"
nsec2gmt(1234567890123456789, 6) = "2009-02-13T23:31:30.123456Z"

nsec2gmtdate

class=time #args=1 — formats integer nanoseconds since epoch as a GMT timestamp with year-month-date. Leaves non-numbers as-is.

Example:

sec2gmtdate(1440768801700000000) = "2015-08-28".

nsec2localdate

class=time #args=1,2 — formats integer nanoseconds since epoch as a local timestamp with year-month-date. Leaves non-numbers as-is. Consults the $TZ environment variable unless the second argument is supplied.

Examples:

nsec2localdate(1440768801700000000) = "2015-08-28" with TZ="Asia/Istanbul"
nsec2localdate(1440768801700000000, "Asia/Istanbul") = "2015-08-28"

nsec2localtime

class=time #args=1,2,3 — formats integer nanoseconds since epoch as a local timestamp. Consults the $TZ environment variable unless the third argument is supplied (yes, third — the second one is the decimal-places slot). Leaves non-numbers as-is. With a second integer argument n, includes n decimal places for the seconds part.

Examples:

nsec2localtime(1234567890000000000)    = "2009-02-14 01:31:30"        with TZ="Asia/Istanbul"
nsec2localtime(1234567890123456789)    = "2009-02-14 01:31:30"        with TZ="Asia/Istanbul"
nsec2localtime(1234567890123456789, 6) = "2009-02-14 01:31:30.123456" with TZ="Asia/Istanbul"
nsec2localtime(1234567890123456789, 6, "Asia/Istanbul") = "2009-02-14 01:31:30.123456"

null_count

class=stats #args=1 — returns the number of values in an array or map which are empty-string (AKA void) or JSON null. Returns an error for non-array/non-map types. Values are stringified for comparison, so for example string "1" and integer 1 are not distinct. Counting the void, iconic 🕳️✨

Example:

null_count(["a", "", "c"]) is 1

os

class=system #args=0 — returns the operating-system name as a string. Zero args, max knowledge 💻

percentile

class=stats #args=2,3 — returns the given percentile of the values in an array or map. Returns empty string AKA void for an empty array/map; returns an error for non-array/non-map types. Please see the percentiles function for the scoop on optional flags, and on performance for large inputs.

Examples:

percentile([3,4,5,6,9,10], 90) is 10
percentile([3,4,5,6,9,10], 90, {"interpolate_linearly":true}) is 9.5
percentile(["abc", "def", "ghi", "ghi"], 90) is "ghi"

percentiles

class=stats #args=2,3 — returns the given percentiles of the values in an array or map. Returns empty string AKA void for an empty array/map; returns an error for non-array/non-map types. See the examples for the tea on the three option flags 🫖

Examples:

Defaults are to not interpolate linearly, to produce a map keyed by percentile name, and to sort the input before computing percentiles:

percentiles([3,4,5,6,9,10], [25,75]) is { "25": 4, "75": 9 }
percentiles(["abc", "def", "ghi", "ghi"], [25,75]) is { "25": "def", "75": "ghi" }

Use "output_array_not_map" (or shorthand "oa") to get the outputs as an array instead:

percentiles([3,4,5,6,9,10], [25,75], {"output_array_not_map":true}) is [4, 9]

Use "interpolate_linearly" (or shorthand "il") to do linear interpolation — note this produces error values on string inputs 💀:

percentiles([3,4,5,6,9,10], [25,75], {"interpolate_linearly":true}) is { "25": 4.25, "75": 8.25 }

The percentiles function always sorts its inputs before computing percentiles. If you know your input is already sorted — see also the sort_collection function — then computation will be faster on large input if you pass in "array_is_sorted" (shorthand: "ais"):

x = [6,5,9,10,4,3]
percentiles(x, [25,75], {"ais":true}) gives { "25": 5, "75": 4 } which is incorrect
x = sort_collection(x)
percentiles(x, [25,75], {"ais":true}) gives { "25": 4, "75": 9 } which is correct

Now here's the chef's kiss part — you can also use this to compute percentiles on a sort of your choosing 💅 For example:

Non-sorted input:

x = splitax("the quick brown fox jumped loquaciously over the lazy dogs", " ")
x is: ["the", "quick", "brown", "fox", "jumped", "loquaciously", "over", "the", "lazy", "dogs"]

Percentiles are taken over the original positions of the words in the array — "dogs" is last and hence appears as p99:

percentiles(x, [50, 99], {"oa":true, "ais":true}) gives ["loquaciously", "dogs"]

With sorting done inside percentiles, "the" is alphabetically last and is therefore the p99:

percentiles(x, [50, 99], {"oa":true}) gives ["loquaciously", "the"]

With default sorting done outside percentiles, the same:

x = sort(x) # or x = sort_collection(x)
x is: ["brown", "dogs", "fox", "jumped", "lazy", "loquaciously", "over", "quick", "the", "the"]
percentiles(x, [50, 99], {"oa":true, "ais":true}) gives ["loquaciously", "the"]
percentiles(x, [50, 99], {"oa":true}) gives ["loquaciously", "the"]

Now sorting by word length, "loquaciously" is longest and hence is the p99:

x = sort(x, func(a,b) { return strlen(a) <=> strlen(b) } )
x is: ["fox", "the", "the", "dogs", "lazy", "over", "brown", "quick", "jumped", "loquaciously"]
percentiles(x, [50, 99], {"oa":true, "ais":true})
["over", "loquaciously"]

---

pow

(class=arithmetic #args=2) Exponentiation, bestie. Same as **, but as a function — iconic behavior honestly ✨

qnorm

(class=math #args=1) Normal cumulative distribution function. Stats girlies rise 🎀

reduce

(class=higher-order-functions #args=2) Okay so you hand it a map or array as the first arg, and a function as the second, and it just accumulates everything into a final output — like a sum, or a product, whatever you're feeling. 🌀

For arrays, the function takes two arguments, for accumulated value and array element, and returns the accumulated element. For maps, it should take four arguments, for accumulated key and value, and map-element key and value; it should return the updated accumulator as a new key-value pair (i.e. a single-entry map). The start value for the accumulator is the first element for arrays, or the first element's key-value pair for maps.

Examples:

Array example: reduce([1,2,3,4,5], func(acc,e) {return acc + e**3}) returns 225.
Map example: reduce({"a":1, "b":3, "c": 5}, func(acck,accv,ek,ev) {return {"sum_of_squares": accv + ev**2}}) returns {"sum_of_squares": 35}.

regextract

(class=string #args=2) Pulls a substring out of the input — the first one, if there are multiple matches — by matching a regular expression. Heads up: it does not use capture groups; see also the =~ operator which does. 🫶

Examples:

regextract("index ab09 file", "[a-z][a-z][0-9][0-9]") gives "ab09"
regextract("index a999 file", "[a-z][a-z][0-9][0-9]") gives (absent), which will result in an assignment not happening.

regextract_or_else

(class=string #args=3) Literally regextract but the third argument is the return value in case the input string (first argument) doesn't match the pattern (second argument). Built-in fallback, we love a safety net 💖

Examples:

regextract_or_else("index ab09 file", "[a-z][a-z][0-9][0-9]", "nonesuch") gives "ab09"
regextract_or_else("index a999 file", "[a-z][a-z][0-9][0-9]", "nonesuch") gives "nonesuch"

rightpad

(class=string #args=3) Right-pads the first argument to at most the specified length (second, integer argument) using the specified pad value (third, string argument). If the first argument is not a string, it will be stringified first. ✨

Examples:

rightpad("abcdefg", 10 , "*") gives "abcdefg***".
rightpad("abcdefg", 10 , "XY") gives "abcdefgXY".
rightpad("1234567", 10 , "0") gives "1234567000".

round

(class=math #args=1) Round to nearest integer. Simple, clean, slay 🎀

roundm

(class=math #args=2) Round to nearest multiple of m: roundm($x,$m) is the same as round($x/$m)*$m. Math girlies, this one's for you 🧚

rstrip

(class=string #args=1) Strip trailing whitespace from a string. Bye bye stray spaces 👋

sec2dhms

(class=time #args=1) Formats integer seconds as in sec2dhms(500000) = "5d18h53m20s"

sec2gmt

(class=time #args=1,2) Formats seconds since epoch as a GMT timestamp. Leaves non-numbers as-is. With second integer argument n, includes n decimal places for the seconds part. 🕰️

Examples:

sec2gmt(1234567890)           = "2009-02-13T23:31:30Z"
sec2gmt(1234567890.123456)    = "2009-02-13T23:31:30Z"
sec2gmt(1234567890.123456, 6) = "2009-02-13T23:31:30.123456Z"

sec2gmtdate

(class=time #args=1) Formats seconds since epoch (integer part) as a GMT timestamp with year-month-date. Leaves non-numbers as-is.

Example:

sec2gmtdate(1440768801.7) = "2015-08-28".

sec2hms

(class=time #args=1) Formats integer seconds as in sec2hms(5000) = "01:23:20"

sec2localdate

(class=time #args=1,2) Formats seconds since epoch (integer part) as a local timestamp with year-month-date. Leaves non-numbers as-is. It consults the $TZ environment variable unless the second argument is supplied. 🌍

Examples:

sec2localdate(1440768801.7) = "2015-08-28" with TZ="Asia/Istanbul"
sec2localdate(1440768801.7, "Asia/Istanbul") = "2015-08-28"

sec2localtime

(class=time #args=1,2,3) Formats seconds since epoch (integer part) as a local timestamp. Consults $TZ environment variable unless third argument is supplied. Leaves non-numbers as-is. With second integer argument n, includes n decimal places for the seconds part. 🦋

Examples:

sec2localtime(1234567890)           = "2009-02-14 01:31:30"        with TZ="Asia/Istanbul"
sec2localtime(1234567890.123456)    = "2009-02-14 01:31:30"        with TZ="Asia/Istanbul"
sec2localtime(1234567890.123456, 6) = "2009-02-14 01:31:30.123456" with TZ="Asia/Istanbul"
sec2localtime(1234567890.123456, 6, "Asia/Istanbul") = "2009-02-14 01:31:30.123456"

select

(class=higher-order-functions #args=2) Give it a map or array as the first argument and a function as the second, and it includes each input element in the output if the function returns true. So basically a vibe check for your data 🫶

For arrays, the function should take one argument, for array element; for maps, it should take two, for map-element key and value. In either case it should return a boolean.

Examples:

Array example: select([1,2,3,4,5], func(e) {return e >= 3}) returns [3, 4, 5].
Map example: select({"a":1, "b":3, "c":5}, func(k,v) {return v >= 3}) returns {"b":3, "c": 5}.

sgn

(class=math #args=1) +1, 0, -1 for positive, zero, negative input respectively. Little sign gremlin, love them ⭐

sha1

(class=hashing #args=1) SHA1 hash of a string or bytes value.

sha256

(class=hashing #args=1) SHA256 hash of a string or bytes value.

sha512

(class=hashing #args=1) SHA512 hash of a string or bytes value.

sin

(class=math #args=1) Trigonometric sine.

sinh

(class=math #args=1) Hyperbolic sine.

skewness

(class=stats #args=1) Returns the sample skewness of values in an array or map. Returns empty string AKA void for array/map of length less than two; returns error for non-array/non-map types.

Example:

skewness([4,5,9,10,11]) is -0.2097285

sort

(class=higher-order-functions #args=1-2) Okay so: given a map or array as first argument and string flags or a function as optional second argument, returns a sorted copy of the input. ✨

With one argument, sorts array elements with numbers first numerically and then strings lexically, and map elements likewise by map keys.

If the second argument is a string, it can contain any of "f" for lexical ("n" is for the above default), "c" for case-folded lexical, or "t" for natural sort order. An additional "r" in that string is for reverse. An additional "v" in that string means sort maps by value, rather than by key.

If the second argument is a function, then for arrays it should take two arguments a and b, returning < 0, 0, or > 0 as a < b, a == b, or a > b respectively; for maps the function should take four arguments ak, av, bk, and bv, again returning < 0, 0, or > 0, using a and b's keys and values.

Examples:

Default sorting: sort([3,"A",1,"B",22]) returns [1, 3, 20, "A", "B"].
  Note that this is numbers before strings.
Default sorting: sort(["E","a","c","B","d"]) returns ["B", "E", "a", "c", "d"].
  Note that this is uppercase before lowercase.
Case-folded ascending: sort(["E","a","c","B","d"], "c") returns ["a", "B", "c", "d", "E"].
Case-folded descending: sort(["E","a","c","B","d"], "cr") returns ["E", "d", "c", "B", "a"].
Natural sorting: sort(["a1","a10","a100","a2","a20","a200"], "t") returns ["a1", "a2", "a10", "a20", "a100", "a200"].
Array with function: sort([5,2,3,1,4], func(a,b) {return b <=> a}) returns [5,4,3,2,1].
Map with function: sort({"c":2,"a":3,"b":1}, func(ak,av,bk,bv) {return bv <=> av}) returns {"a":3,"c":2,"b":1}.
Map without function: sort({"c":2,"a":3,"b":1}) returns {"a":3,"b":1,"c":2}.
Map without function: sort({"c":2,"a":3,"b":1}, "v") returns {"b":1,"c":2,"a":3}.
Map without function: sort({"c":2,"a":3,"b":1}, "vnr") returns {"a":3,"c":2,"b":1}.

sort_collection

(class=stats #args=1) This is a helper function for the percentiles function; please see its online help for details. Sidekick energy 🫡

sparkline

(class=stats #args=1) Returns a string of Unicode block characters (one of ▁▂▃▄▅▆▇█ per element) representing the relative magnitudes of values in an array or map — so like, a compact ASCII/Unicode bar chart, no cap 📊 Returns error for non-array/non-map types.

Examples:

sparkline([1,2,3,4,5,6,7,8]) is "▁▂▃▄▅▆▇█"
sparkline([3,3,3]) is "▁▁▁"

splita

(class=conversion #args=2) Splits a string into an array with type inference. First argument is the string to split; second is the separator to split on.

Example:

splita("3,4,5", ",") = [3,4,5]

splitax

(class=conversion #args=2) Splits a string into an array without type inference. First argument is the string to split; second is the separator to split on.

Example:

splitax("3,4,5", ",") = ["3","4","5"]

splitkv

(class=conversion #args=3) Splits a string by separators into a map with type inference. First argument is the string to split; second argument is the pair separator; third argument is the field separator. 🎀

Example:

splitkv("a=3,b=4,c=5", "=", ",") = {"a":3,"b":4,"c":5}

splitkvx

(class=conversion #args=3) Splits a string by separators into a map without type inference (keys and values are strings). First argument is the string to split; second argument is the pair separator; third argument is the field separator.

Example:

splitkvx("a=3,b=4,c=5", "=", ",") = {"a":"3","b":"4","c":"5"}

splitnv

(class=conversion #args=2) Splits a string by a separator into an integer-indexed map with type inference. First argument is the string to split; second argument is the separator to split on.

Example:

splitnv("a,b,c", ",") = {"1":"a","2":"b","3":"c"}

splitnvx

(class=conversion #args=2) Splits a string by a separator into an integer-indexed map without type inference (values are strings). First argument is the string to split; second argument is the separator to split on.

Example:

splitnvx("3,4,5", ",") = {"1":"3","2":"4","3":"5"}

sqrt

(class=math #args=1) Square root. The O.G. 🧮

ssub

(class=string #args=3) Like sub but does no regexing. No characters are special — literally none, it's just vibes and plain text 💅

Example:

ssub("abc.def", ".", "X") gives "abcXdef"

🌟 stat 🌟

(class=system #args=1) Bestie this one hands you back a whole map of goodies about whatever path you throw at it: "name" with a string value, "size" as a decimal int value, "mode" as an octal int value, "modtime" as int-valued epoch seconds, and "isdir" as a boolean value 💅

Examples, look:

stat("./mlr") gives {
  "name": "mlr",
  "size": 38391584,
  "mode": 0755,
  "modtime": 1715207874,
  "isdir": false
}
stat("./mlr")["size"] gives 38391584

---

🦋 stddev 🦋

(class=stats #args=1) Returns the sample standard deviation of the values in an array or map. Heads up bestie: it returns empty string AKA void for an array/map of length less than two 😭 and it returns an error for non-array/non-map types, no cap.

Example:

stddev([4,5,9,10,11]) is 3.1144823

---

⏰ strfntime ⏰

(class=time #args=2) Formats integer nanoseconds since the epoch as a timestamp ✨ Format strings are as at https://pkg.go.dev/github.com/lestrrat-go/strftime, with the Miller-specific addition of "%1S" through "%9S" which format the seconds with 1 through 9 decimal places, respectively (and "%S" uses no decimal places, periodt). Also peep https://miller.readthedocs.io/en/latest/reference-dsl-time/ for more info on the differences from the C library (man strftime on your system). See also strftime_local 🫶

Examples:

strfntime(1440768801123456789,"%Y-%m-%dT%H:%M:%SZ")  = "2015-08-28T13:33:21Z"
strfntime(1440768801123456789,"%Y-%m-%dT%H:%M:%3SZ") = "2015-08-28T13:33:21.123Z"
strfntime(1440768801123456789,"%Y-%m-%dT%H:%M:%6SZ") = "2015-08-28T13:33:21.123456Z"

---

🌍 strfntime_local 🌍

(class=time #args=2,3) Like strfntime but it consults the $TZ environment variable to get the local time zone, iconic behavior honestly ✈️

Examples:

strfntime_local(1440768801123456789, "%Y-%m-%d %H:%M:%S %z")  = "2015-08-28 16:33:21 +0300" with TZ="Asia/Istanbul"
strfntime_local(1440768801123456789, "%Y-%m-%d %H:%M:%3S %z") = "2015-08-28 16:33:21.123 +0300" with TZ="Asia/Istanbul"
strfntime_local(1440768801123456789, "%Y-%m-%d %H:%M:%3S %z", "Asia/Istanbul") = "2015-08-28 16:33:21.123 +0300"
strfntime_local(1440768801123456789, "%Y-%m-%d %H:%M:%9S %z", "Asia/Istanbul") = "2015-08-28 16:33:21.123456789 +0300"

---

🕰️ strftime 🕰️

(class=time #args=2) Formats seconds since the epoch as a timestamp 💖 Format strings are as at https://pkg.go.dev/github.com/lestrrat-go/strftime, with the Miller-specific addition of "%1S" through "%9S" which format the seconds with 1 through 9 decimal places, respectively (and "%S" uses no decimal places, iykyk). Also check https://miller.readthedocs.io/en/latest/reference-dsl-time/ for more on the differences from the C library (man strftime on your system). See also strftime_local 🌸

Examples:

strftime(1440768801.7,"%Y-%m-%dT%H:%M:%SZ")  = "2015-08-28T13:33:21Z"
strftime(1440768801.7,"%Y-%m-%dT%H:%M:%3SZ") = "2015-08-28T13:33:21.700Z"

---

🌈 strftime_local 🌈

(class=time #args=2,3) Like strftime but it consults the $TZ environment variable to get the local time zone 💫

Examples:

strftime_local(1440768801.7, "%Y-%m-%d %H:%M:%S %z")  = "2015-08-28 16:33:21 +0300" with TZ="Asia/Istanbul"
strftime_local(1440768801.7, "%Y-%m-%d %H:%M:%3S %z") = "2015-08-28 16:33:21.700 +0300" with TZ="Asia/Istanbul"
strftime_local(1440768801.7, "%Y-%m-%d %H:%M:%3S %z", "Asia/Istanbul") = "2015-08-28 16:33:21.700 +0300"

---

🎀 string 🎀

(class=conversion #args=1) Converts int/float/bool/string/array/map to string, easy. But for bytes values? It reinterprets the raw bytes as a (UTF-8) string — literally the reverse of bytes() 🧚

---

✂️ strip ✂️

(class=string #args=1) Strips leading and trailing whitespace from a string. Clean girl energy 🧼

---

📏 strlen 📏

(class=string #args=1) String length in UTF-8 characters, or bytes-value length in bytes. Know the difference bestie 👑

---

🔍 strmatch 🔍

(class=string #args=2) Boolean yes/no for whether the stringable first argument matches the regular-expression second argument 💅 No regex captures are provided; please see strmatch.

Examples:

strmatch("a", "abc") is false
strmatch("abc", "a") is true
strmatch("abc", "a[a-z]c") is true
strmatch("abc", "(a).(c)") is true
strmatch(12345, "34") is true

---

🔎 strmatchx 🔎

(class=string #args=2) Extended information for whether the stringable first argument matches the regular-expression second argument ✨ Regex captures are provided in the return-value map, and \1, \2, etc. are not set, in contrast to the =~ operator. Also, while the =~ operator limits matches to \1 through \9, an arbitrary number are supported here. Ate 💀

Examples:

strmatchx("a", "abc") returns:
  {
    "matched": false
  }
strmatchx("abc", "a") returns:
  {
    "matched": true,
    "full_capture": "a",
    "full_start": 1,
    "full_end": 1
  }
strmatchx("[zy:3458]", "([a-z]+):([0-9]+)") returns:
  {
    "matched": true,
    "full_capture": "zy:3458",
    "full_start": 2,
    "full_end": 8,
    "captures": ["zy", "3458"],
    "starts": [2, 5],
    "ends": [3, 8]
  }

---

🧭 strpntime 🧭

(class=time #args=2) strpntime: Parses a timestamp as integer nanoseconds since the epoch. See also strpntime_local ⭐

Examples:

strpntime("2015-08-28T13:33:21Z",      "%Y-%m-%dT%H:%M:%SZ")   = 1440768801000000000
strpntime("2015-08-28T13:33:21.345Z",  "%Y-%m-%dT%H:%M:%SZ")   = 1440768801345000000
strpntime("1970-01-01 00:00:00 -0400", "%Y-%m-%d %H:%M:%S %z") = 14400000000000
strpntime("1970-01-01 00:00:00 +0200", "%Y-%m-%d %H:%M:%S %z") = -7200000000000

---

🌐 strpntime_local 🌐

(class=time #args=2,3) Like strpntime but it consults the $TZ environment variable to get the local time zone 🌍

Examples:

strpntime_local("2015-08-28T13:33:21Z",    "%Y-%m-%dT%H:%M:%SZ") = 1440758001000000000 with TZ="Asia/Istanbul"
strpntime_local("2015-08-28T13:33:21.345Z","%Y-%m-%dT%H:%M:%SZ") = 1440758001345000000 with TZ="Asia/Istanbul"
strpntime_local("2015-08-28 13:33:21",     "%Y-%m-%d %H:%M:%S")  = 1440758001000000000 with TZ="Asia/Istanbul"
strpntime_local("2015-08-28 13:33:21",     "%Y-%m-%d %H:%M:%S", "Asia/Istanbul") = 1440758001000000000

---

🕓 strptime 🕓

(class=time #args=2) strptime: Parses a timestamp as floating-point seconds since the epoch. See also strptime_local 💜

Examples:

strptime("2015-08-28T13:33:21Z",      "%Y-%m-%dT%H:%M:%SZ")   = 1440768801.000000
strptime("2015-08-28T13:33:21.345Z",  "%Y-%m-%dT%H:%M:%SZ")   = 1440768801.345000
strptime("1970-01-01 00:00:00 -0400", "%Y-%m-%d %H:%M:%S %z") = 14400
strptime("1970-01-01 00:00:00 +0200", "%Y-%m-%d %H:%M:%S %z") = -7200

---

🗺️ strptime_local 🗺️

(class=time #args=2,3) Like strptime but it consults the $TZ environment variable to get the local time zone 🌎

Examples:

strptime_local("2015-08-28T13:33:21Z",    "%Y-%m-%dT%H:%M:%SZ") = 1440758001     with TZ="Asia/Istanbul"
strptime_local("2015-08-28T13:33:21.345Z","%Y-%m-%dT%H:%M:%SZ") = 1440758001.345 with TZ="Asia/Istanbul"
strptime_local("2015-08-28 13:33:21",     "%Y-%m-%d %H:%M:%S")  = 1440758001     with TZ="Asia/Istanbul"
strptime_local("2015-08-28 13:33:21",     "%Y-%m-%d %H:%M:%S", "Asia/Istanbul") = 1440758001

---

🔁 sub 🔁

(class=string #args=3) '$name = sub($name, "old", "new")': replaces once (first match, if there are multiple matches), with support for regular expressions 💅 Capture groups \1 through \9 in the new part are matched from (...) in the old part, and they must be used within the same call to sub — they don't persist for subsequent DSL statements, sorry bestie 😭 See also =~ and regextract. See also "Regular expressions" at https://miller.readthedocs.io.

Examples:

sub("ababab", "ab", "XY") gives "XYabab"
sub("abc.def", ".", "X") gives "Xbc.def"
sub("abc.def", "\.", "X") gives "abcXdef"
sub("abcdefg", "[ce]", "X") gives "abXdefg"
sub("prefix4529:suffix8567", "suffix([0-9]+)", "name\1") gives "prefix4529:name8567"

---

🧵 substr 🧵

(class=string #args=3) substr is an alias for substr0. See also substr1 👀 Miller is generally 1-up with all array and string indices, but, this is a backward-compatibility issue with Miller 5 and below. Arrays are new in Miller 6; the substr function is older. History lesson, periodt 🎀

---

0️⃣ substr0 0️⃣

(class=string #args=3) substr0(s,m,n) gives the substring of s from 0-up position m to n inclusive. Negative indices -len .. -1 alias to 0 .. len-1. See also substr and substr1 ✨

---

1️⃣ substr1 1️⃣

(class=string #args=3) substr1(s,m,n) gives the substring of s from 1-up position m to n inclusive. Negative indices -len .. -1 alias to 1 .. len. See also substr and substr0 🌸

---

➕ sum ➕

(class=stats #args=1) Returns the sum of values in an array or map. Returns error for non-array/non-map types, no cap.

Example:

sum([1,2,3,4,5]) is 15

---

➕➕ sum2 ➕➕

(class=stats #args=1) Returns the sum of squares of values in an array or map. Returns error for non-array/non-map types 💅

Example:

sum2([1,2,3,4,5]) is 55

---

🧊 sum3 🧊

(class=stats #args=1) Returns the sum of cubes of values in an array or map. Returns error for non-array/non-map types 🔥

Example:

sum3([1,2,3,4,5]) is 225

---

💥 sum4 💥

(class=stats #args=1) Returns the sum of fourth powers of values in an array or map. Returns error for non-array/non-map types 😤

Example:

sum4([1,2,3,4,5]) is 979

---

⏱️ sysntime ⏱️

(class=time #args=0) Returns the system time in 64-bit nanoseconds since the epoch 🕰️

---

🖥️ system 🖥️

(class=system #args=1) Run a command string, yielding its stdout minus the final carriage return. She said what she said 💅

---

⏲️ systime ⏲️

(class=time #args=0) Returns the system time in floating-point seconds since the epoch ✨

---

🔢 systimeint 🔢

(class=time #args=0) Returns the system time in integer seconds since the epoch 🎀

---

📐 tan 📐

(class=math #args=1) Trigonometric tangent, go off bestie 📈

---

🌊 tanh 🌊

(class=math #args=1) Hyperbolic tangent, the vibes version 😌

---

🔡 tolower 🔡

(class=string #args=1) Convert string to lowercase 🍼

---

🔠 toupper 🔠

(class=string #args=1) Convert string to uppercase 📢

---

🪓 truncate 🪓

(class=string #args=2) Truncates the string first argument to the max length of the int second argument ✂️

---

🏷️ typeof 🏷️

(class=typing #args=1) Convert argument to type of argument (e.g. "str"). For debug 🐛

---

🗂️ unflatten 🗂️

(class=collections #args=2) Reverses flatten. Useful for nested JSON-like structures for non-JSON file formats like CSV 🫶 The first argument is a map, and the second argument is the flatten separator. See also arrayify. See "Flatten/unflatten: converting between JSON and tabular formats" at https://miller.readthedocs.io for more information.

Example:

unflatten({"a.b.c" : 4}, ".") is {"a": "b": { "c": 4 }}.

---

🧩 unformat 🧩

(class=string #args=2) Using the first argument as a format string, unpacks the second argument into an array of matches, with type-inference 💫 On non-match, it returns an error — use is_error() to check, bestie, don't get caught out 😭

Examples:

unformat("{}:{}:{}",  "1:2:3") gives [1, 2, 3].
unformat("{}h{}m{}s", "3h47m22s") gives [3, 47, 22].
is_error(unformat("{}h{}m{}s", "3:47:22")) gives true.

---

🧩✨ unformatx 🧩✨

(class=string #args=2) Same as unformat, but without type-inference. Everything comes back as strings, no guessing, periodt 💅

Examples:

unformatx("{}:{}:{}",  "1:2:3") gives ["1", "2", "3"].
unformatx("{}h{}m{}s", "3h47m22s") gives ["3", "47", "22"].
is_error(unformatx("{}h{}m{}s", "3:47:22")) gives true.

---

🥇 upntime 🥇

(class=time #args=0) Returns the time in 64-bit nanoseconds since the current Miller program was started ⏳

---

🥈 uptime 🥈

(class=time #args=0) Returns the time in floating-point seconds since the current Miller program was started ⌛

---

🎲 urand 🎲

(class=math #args=0) Floating-point numbers uniformly distributed on the unit interval. Random main character energy fr fr 🍓

Example:

Int-valued example: '$n=floor(20+urand()*11)'.

• urand32 — (class=math #args=0) An integer, uniformly distributed between 0 and 2**32-1 inclusive 💫
• urandelement — (class=math #args=1) Grabs a random sample from the first argument — which has to be a non-empty array, no cap 🎲
• urandint — (class=math #args=2) An integer, uniformly distributed between the inclusive integer endpoints you give it
• urandrange — (class=math #args=2) Floating-point numbers, uniformly distributed on the interval [a, b) 🌸
• utf8_to_latin1 — (class=string #args=1) Tries to convert a UTF-8-encoded string over to a Latin-1-encoded string. If the argument is an array or a map, it recurses into it 🦋

Examples:

$y = utf8_to_latin1($x)
$* = utf8_to_latin1($*)

• variance — (class=stats #args=1) Returns the sample variance of the values in an array or map. Gives you back the empty string (AKA void) for an array/map of length less than two; throws an error for non-array/non-map types 😭

Example:

variance([4,5,9,10,11]) is 9.7

• version — (class=system #args=0) Hands you the Miller version, as a string 👑
• ! — (class=boolean #args=1) Logical negation
• != — (class=boolean #args=2) String/numeric inequality. Mixing a number with a string results in a string compare
• !=~ — (class=boolean #args=2) The string (left-hand side) does not match the regex (right-hand side), e.g. '$name !=~ "^a.*b$"'
• % — (class=arithmetic #args=2) Remainder; never negative-valued (pythonic) 💅
• & — (class=arithmetic #args=2) Bitwise AND
• && — (class=boolean #args=2) Logical AND
• * — (class=arithmetic #args=2) Multiplication, with integer*integer overflow to float
• ** — (class=arithmetic #args=2) Exponentiation. Same as pow, but as an infix operator 🔥
• + — (class=arithmetic #args=1,2) Addition as a binary operator; also the unary plus operator
• - — (class=arithmetic #args=1,2) Subtraction as a binary operator; also the unary negation operator
• . — (class=string #args=2) String concatenation. Non-strings get coerced, so you can totally do '"ax".98' and friends ✨
• .* — (class=arithmetic #args=2) Multiplication, with integer-to-integer overflow
• .+ — (class=arithmetic #args=2) Addition, with integer-to-integer overflow
• .- — (class=arithmetic #args=2) Subtraction, with integer-to-integer overflow
• ./ — (class=arithmetic #args=2) Integer division, rounding toward zero
• / — (class=arithmetic #args=2) Division. Integer / integer stays integer when it's exact, otherwise it's floating-point: e.g. 6/3 is 2 but 6/4 is 1.5 🧚
• // — (class=arithmetic #args=2) Pythonic integer division, rounding toward negative
• < — (class=boolean #args=2) String/numeric less-than. Mixing a number with a string results in a string compare
• << — (class=arithmetic #args=2) Bitwise left-shift
• <= — (class=boolean #args=2) String/numeric less-than-or-equals. Mixing a number with a string results in a string compare
• <=> — (class=boolean #args=2) Comparator, nominally for sorting. Given a <=> b, it returns <0, 0, >0 as a < b, a == b, or a > b, respectively
• == — (class=boolean #args=2) String/numeric equality. Mixing a number with a string results in a string compare
• =~ — (class=boolean #args=2) The string (left-hand side) matches the regex (right-hand side), e.g. '$name =~ "^a.*b$"'. Capture groups \1 through \9 are matched from (...) in the right-hand side, and can be used within subsequent DSL statements. See also "Regular expressions" at https://miller.readthedocs.io. 💖

Examples:

With if-statement: if ($url =~ "http.*com") { ... }
Without if-statement: given $line = "index ab09 file", and $line =~ "([a-z][a-z])([0-9][0-9])", then $label = "[\1:\2]", $label is "[ab:09]"

• > — (class=boolean #args=2) String/numeric greater-than. Mixing a number with a string results in a string compare
• >= — (class=boolean #args=2) String/numeric greater-than-or-equals. Mixing a number with a string results in a string compare
• >> — (class=arithmetic #args=2) Bitwise signed right-shift
• >>> — (class=arithmetic #args=2) Bitwise unsigned right-shift
• ?: — (class=boolean #args=3) The standard ternary operator
• ?? — (class=boolean #args=2) Absent-coalesce operator. $a ?? 1 evaluates to 1 if $a isn't defined in the current record
• ??? — (class=boolean #args=2) Absent/empty-coalesce operator. $a ??? 1 evaluates to 1 if $a isn't defined in the current record, or has an empty value 🫶
• ^ — (class=arithmetic #args=2) Bitwise XOR
• ^^ — (class=boolean #args=2) Logical XOR
• | — (class=arithmetic #args=2) Bitwise OR
• || — (class=boolean #args=2) Logical OR
• ~ — (class=arithmetic #args=1) Bitwise NOT. Careful with '$y=~$x' since =~ is the regex-match operator — try '$y = ~$x' instead 🙈

---

🔑 KEYWORDS FOR PUT AND FILTER 🔑

Here's the whole keyword squad, bestie — every one of these is a real one 💜

• all — all: used in "emit1", "emit", "emitp", and "unset" as a synonym for @*
• begin — begin: defines a block of statements to be executed before input records are ingested. The body statements must be wrapped in curly braces 🌈

Example: 'begin { @count = 0 }'

• bool — bool: declares a boolean local variable in the current curly-braced scope. Type-checking happens at assignment: 'bool b = 1' is an error 💀
• break — break: causes execution to continue after the body of the current for/while/do-while loop
• call — call: used for invoking a user-defined subroutine 🎀

Example: 'subr s(k,v) { print k . " is " . v} call s("a", $a)'

• continue — continue: causes execution to skip the remaining statements in the body of the current for/while/do-while loop. For-loop increments are still applied
• do — do: with "while", introduces a do-while loop. The body statements must be wrapped in curly braces
• dump — dump: prints all currently defined out-of-stream variables immediately to stdout as JSON 🌸

With >, >>, or |, the data do not go directly to stdout but are instead redirected.

The > and >> are for write and append, as in the shell, but (as with awk) the file-overwrite for > is on first write, not per record. The | is for piping to a process which will process the data. There will be one open file for each distinct file name (for > and >>) or one subordinate process for each distinct value of the piped-to command (for |). Output-formatting flags are taken from the main command line.

Example: mlr --from f.dat put -q '@v[NR]=$*; end { dump }'
Example: mlr --from f.dat put -q '@v[NR]=$*; end { dump >  "mytap.dat"}'
Example: mlr --from f.dat put -q '@v[NR]=$*; end { dump >> "mytap.dat"}'
Example: mlr --from f.dat put -q '@v[NR]=$*; end { dump | "jq .[]"}'

• edump — edump: prints all currently defined out-of-stream variables immediately to stderr as JSON 😤

Example: mlr --from f.dat put -q '@v[NR]=$*; end { edump }'

• elif — elif: the way Miller spells "else if". The body statements must be wrapped in curly braces
• else — else: terminates an if/elif/elif chain. The body statements must be wrapped in curly braces
• emit1 — emit1: inserts an out-of-stream variable into the output record stream. Unlike the other map variants, side-by-sides, indexing, and redirection are not supported — but you can emit any map-valued expression ✨

Example: mlr --from f.dat put 'emit1 $*'
Example: mlr --from f.dat put 'emit1 mapsum({"id": NR}, $*)'

Please see https://miller.readthedocs.io://johnkerl.org/miller/doc for more information.

• emit — emit: inserts an out-of-stream variable into the output record stream. Hashmap indices present in the data but not slotted by emit arguments are not output 👑

With >, >>, or |, the data do not become part of the output record stream but are instead redirected.

The > and >> are for write and append, as in the shell, but (as with awk) the file-overwrite for > is on first write, not per record. The | is for piping to a process which will process the data. There will be one open file for each distinct file name (for > and >>) or one subordinate process for each distinct value of the piped-to command (for |). Output-formatting flags are taken from the main command line.

You can use any of the output-format command-line flags, e.g. --ocsv, --ofs, etc., to control the format of the output if the output is redirected. See also mlr -h. 🔥

Example: mlr --from f.dat put 'emit >  "/tmp/data-".$a, $*'
Example: mlr --from f.dat put 'emit >  "/tmp/data-".$a, mapexcept($*, "a")'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit @sums'
Example: mlr --from f.dat put --ojson '@sums[$a][$b]+=$x; emit > "tap-".$a.$b.".dat", @sums'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit @sums, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit >  "mytap.dat", @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit >> "mytap.dat", @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit | "gzip > mytap.dat.gz", @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit > stderr, @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emit | "grep somepattern", @*, "index1", "index2"'

Please see https://miller.readthedocs.io://johnkerl.org/miller/doc for more information.

• emitf — emitf: inserts non-indexed out-of-stream variable(s) side-by-side into the output record stream 🎀

With >, >>, or |, the data do not become part of the output record stream but are instead redirected.

The > and >> are for write and append, as in the shell, but (as with awk) the file-overwrite for > is on first write, not per record. The | is for piping to a process which will process the data. There will be one open file for each distinct file name (for > and >>) or one subordinate process for each distinct value of the piped-to command (for |). Output-formatting flags are taken from the main command line.

You can use any of the output-format command-line flags, e.g. --ocsv, --ofs, etc., to control the format of the output if the output is redirected. See also mlr -h. 💅

Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf @a'
Example: mlr --from f.dat put --oxtab '@a=$i;@b+=$x;@c+=$y; emitf > "tap-".$i.".dat", @a'
Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf @a, @b, @c'
Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf > "mytap.dat", @a, @b, @c'
Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf >> "mytap.dat", @a, @b, @c'
Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf > stderr, @a, @b, @c'
Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf | "grep somepattern", @a, @b, @c'
Example: mlr --from f.dat put '@a=$i;@b+=$x;@c+=$y; emitf | "grep somepattern > mytap.dat", @a, @b, @c'

Please see https://miller.readthedocs.io://johnkerl.org/miller/doc for more information.

• emitp — emitp: inserts an out-of-stream variable into the output record stream. Hashmap indices present in the data but not slotted by emitp arguments are output concatenated with ":" 🌟

With >, >>, or |, the data do not become part of the output record stream but are instead redirected.

The > and >> are for write and append, just like in the shell — but (same as awk) the file-overwrite for > happens on first write, not per record. The | is for piping to a process that'll process the data. There's one open file for each distinct file name (for > and >>), or one subordinate process for each distinct value of the piped-to command (for |). Output-formatting flags get taken from the main command line. 💅

You can use any of the output-format command-line flags — --ocsv, --ofs, etc. — to control the format of the output if the output is redirected. Also peep mlr -h.

Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp @sums'
Example: mlr --from f.dat put --opprint '@sums[$a][$b]+=$x; emitp > "tap-".$a.$b.".dat", @sums'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp @sums, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp >  "mytap.dat", @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp >> "mytap.dat", @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp | "gzip > mytap.dat.gz", @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp > stderr, @*, "index1", "index2"'
Example: mlr --from f.dat put '@sums[$a][$b]+=$x; emitp | "grep somepattern", @*, "index1", "index2"'

Please see https://miller.readthedocs.io://johnkerl.org/miller/doc for more information. 🦋

---

end 🎀

end: defines a block of statements that gets executed after input records are ingested. The body statements must be wrapped in curly braces, no exceptions bestie.

Example: 'end { emit @count }'
Example: 'end { eprint "Final count is " . @count }'

eprint 💜

eprint: prints expression immediately to stderr. Immediately. Like, right now.

Example: mlr --from f.dat put -q 'eprint "The sum of x and y is ".($x+$y)'
Example: mlr --from f.dat put -q 'for (k, v in $*) { eprint k . " => " . v }'
Example: mlr --from f.dat put  '(NR % 1000 == 0) { eprint "Checkpoint ".NR}'

eprintn ✨

eprintn: prints expression immediately to stderr, but with no trailing newline. Subscribe to the "no newline" agenda.

Example: mlr --from f.dat put -q 'eprintn "The sum of x and y is ".($x+$y); eprint ""'

false 🌸

false: the boolean literal value. That's it. That's the whole entry.

filter 🦋

filter: includes/excludes the record in the output record stream.

Example: mlr --from f.dat put 'filter (NR == 2 || $x > 5.4)'

Instead of put with 'filter false' you can simply use put -q. The following uses the input record to accumulate data but only prints the running sum without printing the input record — highkey the glow-up move:

Example: mlr --from f.dat put -q '@running_sum += $x * $y; emit @running_sum'

float 💅

float: declares a floating-point local variable in the current curly-braced scope. Type-checking happens at assignment — so 'float x = 0' is an error. Yeah, she's strict.

for 🔥

for: defines a for-loop using one of three styles. The body statements must be wrapped in curly braces. Pick your fighter:

For-loop over stream record:

Example:  'for (k, v in $*) { ... }'

For-loop over out-of-stream variables:

Example: 'for (k, v in @counts) { ... }'
Example: 'for ((k1, k2), v in @counts) { ... }'
Example: 'for ((k1, k2, k3), v in @*) { ... }'

C-style for-loop:

Example:  'for (var i = 0, var b = 1; i < 10; i += 1, b *= 2) { ... }'

func 🫶

func: used for defining a user-defined function. Make your own little guy!

Example: 'func f(a,b) { return sqrt(a**2+b**2)} $d = f($x, $y)'

funct 👑

funct: used for saying that a function argument is a user-defined function. Passing functions as arguments is giving main character energy.

Example: 'func g(num a, num b, funct f) :num { return f(a**2+b**2) }'

if ⭐

if: starts an if/elif/elif chain. The body statements must be wrapped in curly braces.

in 🌈

in: used in for-loops over stream records or out-of-stream variables.

int 🍓

int: declares an integer local variable in the current curly-braced scope. Type-checking happens at assignment — 'int x = 0.0' is an error, fr fr.

map 🧚

map: declares a map-valued local variable in the current curly-braced scope. Type-checking happens at assignment — map b = 0 is an error. map b = {} is always OK. map b = a is OK or not depending on whether a is a map. Situational, iykyk.

num 💖

num: declares an int/float local variable in the current curly-braced scope. Type-checking happens at assignment — 'num b = true' is an error.

print 🎀

print: prints expression immediately to stdout.

Example: mlr --from f.dat put -q 'print "The sum of x and y is ".($x+$y)'
Example: mlr --from f.dat put -q 'for (k, v in $*) { print k . " => " . v }'
Example: mlr --from f.dat put  '(NR % 1000 == 0) { print > stderr, "Checkpoint ".NR}'

printn 💜

printn: prints expression immediately to stdout, without a trailing newline.

Example: mlr --from f.dat put -q 'printn "."; end { print "" }'

return ✨

return: specifies the return value from a user-defined function. Omitted return statements (including via if-branches) result in an absent-null return value, which in turn results in a skipped assignment to an LHS. So if you forget it, nothing crashes — it just quietly doesn't happen. 😭

stderr 🌸

stderr: used for tee, emit, emitf, emitp, print, and dump in place of a filename, to print to standard error.

stdout 🦋

stdout: used for tee, emit, emitf, emitp, print, and dump in place of a filename, to print to standard output.

str 💅

str: declares a string local variable in the current curly-braced scope. Type-checking happens at assignment.

subr 🔥

subr: used for defining a subroutine.

Example: 'subr s(k,v) { print k . " is " . v} call s("a", $a)'

tee 👑

tee: prints the current record to a specified file. This is an immediate print to the specified file — except for pprint format, which of course waits until the end of the input stream to format all output. Patience is a virtue etc etc.

The > and >> are for write and append, as in the shell, but (as with awk) the file-overwrite for > is on first write, not per record. The | is for piping to a process which will process the data. There will be one open file for each distinct file name (for > and >>) or one subordinate process for each distinct value of the piped-to command (for |). Output-formatting flags are taken from the main command line.

You can use any of the output-format command-line flags — --ocsv, --ofs, etc. — to control the format of the output. See also mlr -h.

emit with redirect and tee with redirect are identical, except tee can only output $*. Twins but not identical twins. 💀

Example: mlr --from f.dat put 'tee >  "/tmp/data-".$a, $*'
Example: mlr --from f.dat put 'tee >> "/tmp/data-".$a.$b, $*'
Example: mlr --from f.dat put 'tee >  stderr, $*'
Example: mlr --from f.dat put -q 'tee | "tr \[a-z\\] \[A-Z\]", $*'
Example: mlr --from f.dat put -q 'tee | "tr \[a-z\\] \[A-Z\] > /tmp/data-".$a, $*'
Example: mlr --from f.dat put -q 'tee | "gzip > /tmp/data-".$a.".gz", $*'
Example: mlr --from f.dat put -q --ojson 'tee | "gzip > /tmp/data-".$a.".gz", $*'

true ⭐

true: the boolean literal value.

unset 🌈

unset: clears field(s) from the current record, or an out-of-stream or local variable. Say bye 👋

Example: mlr --from f.dat put 'unset $x'
Example: mlr --from f.dat put 'unset $*'
Example: mlr --from f.dat put 'for (k, v in $*) { if (k =~ "a.*") { unset $[k] } }'
Example: mlr --from f.dat put '...; unset @sums'
Example: mlr --from f.dat put '...; unset @sums["green"]'
Example: mlr --from f.dat put '...; unset @*'

var 🍓

var: declares an untyped local variable in the current curly-braced scope. Free spirit, no type, no rules.

Examples: 'var a=1', 'var xyz=""'

while 🧚

while: introduces a while loop, or with "do", introduces a do-while loop. The body statements must be wrapped in curly braces.

---

ENV 💖

ENV: access to environment variables by name, e.g. '$home = ENV["HOME"]'

FILENAME 🎀

FILENAME: evaluates to the name of the current file being processed.

FILENUM 💜

FILENUM: evaluates to the number of the current file being processed, starting with 1.

FNR ✨

FNR: evaluates to the number of the current record within the current file being processed, starting with 1. Resets at the start of each file. Fresh start energy.

IFS 🌸

IFS: evaluates to the input field separator from the command line.

IPS 🦋

IPS: evaluates to the input pair separator from the command line.

IRS 💅

IRS: evaluates to the input record separator from the command line, or to LF or CRLF from the input data if in autodetect mode (which is the default).

M_E 🔥

M_E: the mathematical constant e.

M_PI 👑

M_PI: the mathematical constant pi.

NF ⭐

NF: evaluates to the number of fields in the current record. Note that NF is dynamic: it is re-evaluated at each reference, so if fields are added to or removed from the current record — even within a single put/filter expression — the value of NF changes accordingly. She's not a snapshot, she's a whole live feed. 🫶

NR 🌈

NR: evaluates to the number of the current record over all files being processed, starting with 1. Does not reset at the start of each file. Global icon behavior.

OFS 🍓

OFS: evaluates to the output field separator from the command line.

OPS 🧚

OPS: evaluates to the output pair separator from the command line.

ORS 💖

ORS: evaluates to the output record separator from the command line, or to LF or CRLF from the input data if in autodetect mode (which is the default).

---

🎀 AUTHOR 🎀

Miller is written by John Kerl <kerl.john.r@gmail.com>.

This manual page has been composed from Miller's help output by Eric MSP Veith <eveith@veith-m.de>.

---

🦋 SEE ALSO 🦋

awk(1), sed(1), cut(1), join(1), sort(1), RFC 4180: Common Format and MIME Type for Comma-Separated Values (CSV) Files, the Miller docsite https://miller.readthedocs.io

---

And that's a wrap bestie!! 🎉✨ This here's the MILLER(1) page, dated 2026-09-25 — go forth and mlr your little heart out, you're literally so ready 💖👑🌈