GRCOLLECT(1)

👑 NAME 👑

okay so bestie, welcome to grcollect — section 1 of the manual, the user commands shelf, the one where the good stuff lives 💅

grcollect — performing transposition on the input tabulated data. that's it, that's the whole vibe. tabulated data going in, transposed data coming out ✨

---

🌸 SYNOPSIS 🌸

grcollect [options] <input> [...] [-o <output>|-b <basename>]

look at her. short. iconic. does exactly what it says 🎀

---

🦋 DESCRIPTION 🦋

okay so grcollect is lowkey doing two things, and both of them are a whole personality 💖

the first thing — data transposition. the input (which gets read from files or from standard input, either way bestie) is sorted and splitted into separate files, and the splitting is based on a respective key. where do those keys come from? the input data itself, obviously 💅. and in the specific case where the input comes from more files and each key is unique in a given file, that process is what we call data transposition — because it's basically the same energy as when a 2 dimensional data matrix is stored in the form where each row lives in a separate file, and you wanna transpose the matrix, i.e. store each column in a separate file. same thing, different fonts ✨

the other thing — some sort of statistics on the data that's associated with different keys. and when we say statistics we mean: average (mean, median, mode) and scatter (standard deviation or median deviance) estimations, with the optional deselection of outlier points, plus summation, count statistics, and so on. so many numbers. so many vibes 🧚

---

✨ OPTIONS ✨

💫 General options:

• -h, --help — give general summary about the command line options. quick lil hello 🫶
• --long-help, --help-long — gives a detailed list of command line options. the extended director's cut 👑
• --wiki-help, --help-wiki, --mediawiki-help, --help-mediawiki — gives a detailed list of command line options in Mediawiki format. for when you want your help docs to look cute on a wiki 🌈
• --version, --version-short, --short-version — give some version information about the program. the "who am i" moment 💜
• <input> [,<input>, ...] — name of the input file. at least, one file should be specified. reading from standard input can be forced using a single dash - as input file name. more dashes are silently ignored. (so - is the move, but ------ is just extra, bestie, she's not gonna yell at you 🙈)
• -c, --col-base <key column index> — column index for the key. this is where the key lives, point at it 🔥

🎀 Data transposition specific options:

• -b, --basename <base-%b-name> — base name of the output files. the base name string should conatain at least one %b tag, which is replaced by the respective key string on the creation of the file. (yes, "conatain", the original said it like that and we're keeping every crumb of it 💀)
• -x, --extension <extension>, -p, --prefix <prefix> — equivalent to -b|--basename <prefix>%b.<extension>. note that in practice, <prefix> might be some sort of directory name and extension is a regular file extension, but the above substitution is done literally. therefore, the "dot" between the key and the <extension> is always inserted in the final name of the output files, but a trailing slash is required at the end of <prefix> if the files are to be created in that particular directory. also note that in this case, the target directory must exist before the invocation of grcollect, otherwise the output files cannot be created. so like, make the folder first, okay?? 😭
• -C, --comment — insert a commented line (starting with #) containing information about the version and command line invocation syntax of grcollect to the beginning of the transposed files. receipts, basically 🧾
• -S, --additional-comment <...> — insert an additional commented lines (starting with #) to the beginning of the transposed files. more notes, more lore 💜

⭐ Options for cumulative statistics:

• -d, --col-stat <>[,...] — comma-separated list of column indices on which the statistics are to be calculated. columns with non-numerical contents are ignored. note that this option imply the cumulative statistics mode of grcollect. so like, flipping the switch, periodt ✨
• -o, --output <filename> — the name of the output file to which the output statistics are written. the total number of columns in this file will be 1+C*N, where C is the number of columns (see -d|--col-stat) on which the statistics are calculated and N is the number of statistic quantities (see --stat). the first column in the output file is the key, which is followed by the per-column list of statistics, in the same order as the user defined after -d|--col-stat and --stat. math girlies rise 🙌
• -s, --stat <list of statistics> — comma-separated list of statistics to be estimated on the input data. these can be one or more of the following:
  • count — total number of records, for the given key.
  • rcount — the number of records after rejecting outliers (i.e. it is always the same as the count value if no --rejection was used).
  • mean, median, mode — mean, median or mode statistics of the data.
  • rmean, rmedian, rmode — mean, median or mode, after rejecting outliers.
  • {mean|median|mode}stddev, {mean|median|mode}meddev, stddev — scatter of the data around the mean, median or mode. the scatter can either be standard deviation (stddev) or median deviance (meddev). the literal stddev is the classic standard deviation, equivalent to meanstddev.
  • r{mean|median|mode}stddev, r{mean|median|mode}meddev, rstddev — the same scatters as above but after rejecting outliers.
  • sum, rsum — sum of the data, esp. total sum and sum after rejecting outliers.
  • sum2, rsum2 — sum of the squares, total and after rejecting outliers.
  • min, max — minimal and maximal data values.
  • rmin, rmax — minimal and maximal data values after the rejection of outliers.
• -r, --rejection column=<index>,<rejection parameters> — comma-separated directives for outlier rejection for the specified column. the rejection parameters are:
  • iterations=<n> — maximum number of iterations to reject outliers.
  • mean, median, mode — use the mean, median or mode for the center of the rejection.
  • stddev, meddev, absolute=<limit> — use the standard deviation or median deviance for rejection limit units or define an absolute limit for rejection level.

note that each column can have different kind of rejection method, thus more than one --rejection ... command line option can be used at the invocation of grcollect. mix and match, bestie, she's not judging 💅

🌈 Other options:

• -m, --max-memory <memory>[kmg] — maximum amount of memory available for grcollect. the prefixes k, m or g can be used for kilobytes, megabytes and gigabytes, respectively. on 32bit systems, the maximum memory is limited to 3gigabytes. note that grcollect does not use any kind of operating system specific methods to determine the maximum amount of memory, it always should be set by the user. the default value of 8 megabytes is somewhat small, so upon massive data transposition (tens or hundreds of gigabytes), this limit is worth to be set accordingly to the physical memory available. so like… set it, bestie, don't just leave her on 8 megs 😭
• -t, --tmpdir <directory> — directory for temporary file storage. note that the default temporary directory is always the current one (which is equivalent to define --tmpdir ./), since in a usual configuration the /tmp directory is small, moreover, it can be some sort of "tmpfs", temporary file system mount on the physical memory itself. she's keeping her temp files close to home, and honestly? valid 🫶

---

💀 REPORTING BUGS 💀

found something cursed? report bugs to <apal@szofi.net>, see also https://fitsh.net/ 🐛

---

💖 COPYRIGHT 💖

Copyright © 1996, 2002, 2004-2008, 2010-2016, 2018-2020; Pal, Andras <apal@szofi.net> 👑

---

and that's the whole page, bestie 🎀 this was grcollect 0.9.4 dated 2021.01.24, from the January 2021 era of the GRCOLLECT(1) manual — go transpose something and slay 💅✨