CSVSTAT(1)

📦
version 2.2.0 · 7 versions
, in arch rolling, nixos 26.05, nixpkgs unstable, ubuntu resolute

💖 CSVSTAT(1) — the csvkit glow-up 💖

okay bestie, sit down, hydrate, because this is csvstat from csvkit and she is literally the main character of your data pipeline. 🌈

✨ NAME ✨

csvstat — csvstat Documentation 🎀

🌸 DESCRIPTION 🌸

So here's the vibe: csvstat prints descriptive statistics for all columns in a CSV file. And the way she does it?? Iconic. She will intelligently determine the type of each column and then print analysis relevant to that type — ranges for dates, mean and median for integers, etc. She's not doing the same boring thing to every column, she's reading the room. 💅

Check the usage, it's giving full menu:

usage: csvstat [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b]
               [-p ESCAPECHAR] [-z FIELD_SIZE_LIMIT] [-e ENCODING] [-L LOCALE]
               [-S] [--blanks] [--null-value NULL_VALUES [NULL_VALUES ...]]
               [--date-format DATE_FORMAT] [--datetime-format DATETIME_FORMAT]
               [-H] [-K SKIP_LINES] [-v] [-l] [--zero] [-V] [--csv] [--json]
               [-i INDENT] [-n] [-c COLUMNS] [--type] [--nulls] [--non-nulls]
               [--unique] [--min] [--max] [--sum] [--mean] [--median]
               [--stdev] [--len] [--max-precision] [--freq]
               [--freq-count FREQ_COUNT] [--count]
               [--decimal-format DECIMAL_FORMAT] [-G] [-y SNIFF_LIMIT] [-I]
               [FILE]

Print descriptive statistics for each column in a CSV file.

positional arguments:
  FILE                  The CSV file to operate on. If omitted, will accept
                        input as piped data via STDIN.

optional arguments:
  -h, --help            show this help message and exit
  --csv                 Output results as a CSV table, rather than plain text.
  --json                Output results as JSON text, rather than plain text.
  -i INDENT, --indent INDENT
                        Indent the output JSON this many spaces. Disabled by
                        default.
  -n, --names           Display column names and indices from the input CSV
                        and exit.
  -c COLUMNS, --columns COLUMNS
                        A comma-separated list of column indices, names or
                        ranges to be examined, e.g. "1,id,3-5". Defaults to
                        all columns.
  --type                Only output data type.
  --nulls               Only output whether columns contains nulls.
  --non-nulls           Only output counts of non-null values.
  --unique              Only output counts of unique values.
  --min                 Only output smallest values.
  --max                 Only output largest values.
  --sum                 Only output sums.
  --mean                Only output means.
  --median              Only output medians.
  --stdev               Only output standard deviations.
  --len                 Only output the length of the longest values.
  --max-precision       Only output the most decimal places.
  --freq                Only output lists of frequent values.
  --freq-count FREQ_COUNT
                        The maximum number of frequent values to display.
  --count               Only output total row count.
  --decimal-format DECIMAL_FORMAT
                        %-format specification for printing decimal numbers.
                        Defaults to locale-specific formatting with "%.3f".
  -G, --no-grouping-separator
                        Do not use grouping separators in decimal numbers.
  -y SNIFF_LIMIT, --snifflimit SNIFF_LIMIT
                        Limit CSV dialect sniffing to the specified number of
                        bytes. Specify "0" to disable sniffing entirely, or
                        "-1" to sniff the entire file.
  -I, --no-inference    Disable type inference (and --locale, --date-format,
                        --datetime-format, --no-leading-zeroes) when parsing
                        the input.

🫶 See also: Arguments common to all tools. (all those common flags in the usage line up there — the -d, -t, -q, -u, -b, -p, -z, -e, -L, -S, --blanks, --null-value, --date-format, --datetime-format, -H, -K, -v, -l, --zero, -V girlies — they live over there, go say hi.)

💅 OPTIONS 💅

okay so here's every single one, no skips, no cuts, we're thorough in this house 🎀

  • -h, --help — shows this help message and exits. the classic. the moment of "wait what does this do again". 🙈
  • --csv — spits the results out as a CSV table instead of plain text. for when you want to feed her back into another tool.
  • --json — spits the results out as JSON text instead of plain text. machine-readable slay. 🔥
  • -i INDENT, --indent INDENT — indents the output JSON by this many spaces. Disabled by default, so no indent unless you ask.
  • -n, --names — shows the column names and indices from the input CSV, then exits. a lil table of contents before the real thing. 👑
  • -c COLUMNS, --columns COLUMNS — a comma-separated list of column indices, names or ranges to be examined, e.g. "1,id,3-5". Defaults to all columns. pick your faves.
  • --type — only output the data type. that's it. that's the whole thing.
  • --nulls — only output whether the columns contain nulls. yes/no energy.
  • --non-nulls — only output counts of non-null values. the present ones. we love a reliable turnout.
  • --unique — only output counts of unique values. everyone who's one of a kind. 💜
  • --min — only output the smallest values. tiny icons.
  • --max — only output the largest values. big main character energy.
  • --sum — only output sums. math o'clock.
  • --mean — only output means. average girlies rise.
  • --median — only output medians. the middle child, finally getting attention.
  • --stdev — only output standard deviations. how spread out everyone is.
  • --len — only output the length of the longest values. who's the wordiest in the column.
  • --max-precision — only output the most decimal places. how far past the dot we're going.
  • --freq — only output lists of frequent values. the popular kids.
  • --freq-count FREQ_COUNT — the maximum number of frequent values to display. keeps the list from becoming a whole novel.
  • --count — only output the total row count. just the number, ma'am.
  • --decimal-format DECIMAL_FORMAT — a %-format specification for printing decimal numbers. Defaults to locale-specific formatting with "%.3f".
  • -G, --no-grouping-separator — do not use grouping separators in decimal numbers. goodbye 1,000, hello 1000. 💀
  • -y SNIFF_LIMIT, --snifflimit SNIFF_LIMIT — limits CSV dialect sniffing to the specified number of bytes. Specify "0" to disable sniffing entirely, or "-1" to sniff the entire file.
  • -I, --no-inference — disables type inference (and --locale, --date-format, --datetime-format, --no-leading-zeroes) when parsing the input. she stops guessing and just takes things at face value. no cap.

🦋 EXAMPLES 🦋

Basic use 🍓

just hand her a file and let her cook:

csvstat examples/realdata/FY09_EDU_Recipients_by_State.csv

When a statistic name is passed, only that stat will be printed 🌟

like so:

$ csvstat --min examples/realdata/FY09_EDU_Recipients_by_State.csv
  1. State Name: None
  2. State Abbreviate: None
  3. Code: 1
  4. Montgomery GI Bill-Active Duty: 435
  5. Montgomery GI Bill- Selective Reserve: 48
  6. Dependents' Educational Assistance: 118
  7. Reserve Educational Assistance Program: 60
  8. Post-Vietnam Era Veteran's Educational Assistance Program: 1
  9. TOTAL: 768
 10. j: None

If a single stat and a single column are requested, only a value will be returned 🎀

no labels, no fluff, just the number:

$ csvstat -c 4 --mean examples/realdata/FY09_EDU_Recipients_by_State.csv
6,263.904

👑 AUTHOR 👑

Christopher Groskopf and contributors 🫶 (the icons, the legends)

2016, Christopher Groskopf and James McKinney

and that's the whole page bestie!! csvkit 2.2.0, dated Aug 16, 2024 — go forth, run csvstat, and let the columns tell you their whole life story. no cap, she ate. 💅✨🌸🦋🎀💜

📦 where this page comes from