DBCOLSTATSCORES(1)

📦
version 3.9 · 3 versions
, in epel 9

💅 DBCOLSTATSCORES — the manual, but make it make sense

okay so bestie, the manual name is dbcolstatscores and it lives in the "User Contributed Perl Documentation" section, and honestly? it ate. let's get into it 🫶

✨ NAME ✨

dbcolstatscores — compute z-scores (and optionally t-scores, if you're feeling fancy) for every value in a population. no cap, that's the whole gig.

🦋 SYNOPSIS 🦋

dbcolstatscores [-t] [--tmean=MEAN] [--tstddev=STDDEV] column

🌸 DESCRIPTION 🌸

So the vibe is: this thing computes statistics (a z-score, and optionally a t-score) over a COLUMN of numbers. It creates new columns called zscore and tscore. T-scores are only computed if you ask for them with the -t option, or if --tmean or --tstddev are explicitly specified (the defaults are a mean of 50 and a standard deviation of 10, iykyk).

Now, if you remember literally anything from stats class (no judgment if you don't, bestie, we've all been there 😭): a z-score is just the value normalized by mean and standard deviation, so that 0.0 is the mean and positive or negative values are multiples of the standard deviation. It assumes the data follows a normal (Gaussian) distribution. that's the fine print.

A t-score scales the z-score to match a mean of 50 and a standard deviation of 10. And this program is generous — it lets you do generalized t-scores that use any mean and standard deviation. main character energy honestly 👑

Other scales are sometimes used too! The Wechsler Adult Intelligence Scale (one type of IQ test) is adjusted to a mean of 100 and a standard deviation of 15. Other tests scale to other standard deviations. so many vibes, so many scales.

Heads up 🫶 this program requires two passes over the data, and it consumes O(1) memory and O(number of rows) disk space. efficient queen.

💖 OPTIONS 💖

  • -a or --include-non-numeric — compute stats over all the records (treat non-numeric records as zero rather than just ignoring them). inclusive behavior, we respect it.
  • -t — compute t-scores in addition to z-scores. slay.
  • --tmean MEAN — use the given MEAN for t-scores.
  • --tstddev STDDEV or --tsd STDDEV — use the given STDDEV for the standard deviation of the t-scores. two names, same energy ✨
  • -f FORMAT or --format FORMAT — specify a printf(3)-style format for the output statistics. Defaults to "%.5g".
  • -T TmpDir — where to put tmp files. It also uses the environment variable TMPDIR, if -T is not specified. Default is /tmp.

🎀 the standard fsdb options (it supports these too, obviously) 🎀

  • -d — enable debugging output.
  • -i or --input InputSource — read from InputSource, typically a file name, or "-" for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
  • -o or --output OutputDestination — write to OutputDestination, typically a file name, or "-" for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
  • --autorun or --noautorun — by default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The --(no)autorun option controls that behavior within Perl.
  • --help — show help. we love a helpful icon 💅
  • --man — show the full manual. (hi, that's this 💖)

🧚 SAMPLE USAGE 🧚

🍓 Input:

#fsdb name id test1
a 1 80
b 2 70
c 3 65
d 4 90
e 5 70
f 6 90

⭐ Command:

cat DATA/grades.fsdb | dbcolstatscores --tmean 50 --tstddev 10 test1 | dbcolneaten

🌈 Output:

#fsdb name id test1 zscore:d   tscore:d
a       1  80    0.23063  52.306
b       2  70    -0.69188 43.081
c       3  65    -1.1531  38.469
d       4  90    1.1531   61.531
e       5  70    -0.69188 43.081
f       6  90    1.1531   61.531
#  | dbcolstatscores --tmean 50 --tstddev 10 test1
#  | dbcolneaten

like, look at those z-scores. immaculate. 🔥

👑 SEE ALSO 👑

go read those too, bestie, they're all in the same friend group fr fr.

Copyright (C) 1991-2022 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details.

and that's the whole page 🎀 this was perl v5.32.1, dated 2024-08-02, served to you fresh from DBCOLSTATSCORES(1). mwah 💋 go compute some z-scores, you absolute icon. ✨🦋💅

📦 where this page comes from