DBCOLMOVINGSTATS(1)

📦
version 3.9 · 3 versions
, in epel 9

💅 DBCOLMOVINGSTATS 💅

okay bestie, so this is a page from the User Contributed Perl Documentation section of your manual, and it's literally all about dbcolmovingstats — a lil tool that computes moving statistics over a window of a column of data. icon behavior tbh 🎀

✨ NAME ✨

dbcolmovingstats — compute moving statistics over a window of a column of data. that's it, that's the vibe 🦋

🌸 SYNOPSIS 🌸

dbcolmovingstats [-am] [-w WINDOW] [-e EmptyValue] [-k KEY] column

like, that's the whole thing. short and slay 💅

💖 DESCRIPTION 💖

So the deal is: we compute moving statistics over a COLUMN of data. Records that contain non-numeric data are considered null and they do not contribute to the stats — unless you pass -a, in which case they get treated as zeros instead of being ignored. pick your fighter ✨

Statitics are computed over a WINDOW of samples of data. (yes, girl, the original page really spells it "Statitics" — we're keeping it, it's part of the lore 🫶)

🚧 [In progress 2020-11-12, but not completed: Alternatively, if a key column is given with -k KEY, then a we treat the key column as a time value and compute the time-weighted mean.]

Currently we compute mean and sample standard deviation. (Note we only compute sample standard deviation, not full population — highkey important distinction, don't @ me.) Optionally, with -m we also compute median. (Currently there is no support for generalized quantiles. sad but true 😭)

Values before a sufficient number have been accumulated are given the empty value (if specified with -e). If no empty value is given, stats are computed on as many are possible if no empty value is specified.

Dbcolmovingstats runs in O(1) memory, but must buffer a full window of data. Quantiles currently will repeatedly sort the window and so may perform poorly with wide windows. ngl that's the tradeoff bestie 💀

👑 OPTIONS 👑

  • -a, --include-non-numeric — Compute stats over all records (treat non-numeric records as zero rather than just ignoring them). no record left behind ✨
  • -w, --window WINDOW — WINDOW of how many items to accumulate (defaults to 10). (For compatibility with fsdb-1.x, -n is also supported. iykyk 🧚)
  • -k, --key KEY — The KEY specifies a field that is used to evaluate the window — a window must span at most this range of value so the key field. (For example, if KEY is the time and window is 60, then enough samples will be added to make at most 60s of observations. With a key, sampling can be irregular.) If key is specified, we also output a moving_n field for how many samples are in each window. irregular sampling? she understood the assignment 💅
  • -m, --median — Show median of the window in addition to mean. median girlies rise 🌸
  • -e E, --empty E — Give value E as the value for empty (null) records. This null value is then output before a full window is accumulated.
  • -f FORMAT, --format FORMAT — Specify a printf(3)-style format for output mean and standard deviation. Defaults to "%.5g". dress your numbers up, bestie 🎀

Eventually we expect to support other options of dbcolstats.

🌈 the standard fsdb options (she comes with extras) 🌈

This module also supports the standard fsdb options:

  • -d — Enable debugging output.
  • -i, --input InputSource — Read from InputSource, typically a file name, or "-" for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
  • -o, --output OutputDestination — Write to OutputDestination, typically a file name, or "-" for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.
  • --autorun, --noautorun — By default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The "--(no)autorun" option controls that behavior within Perl.
  • --help — Show help. we love a helper 🫶
  • --man — Show full manual. (you are literally here rn 🔥)

💜 SAMPLE USAGE 💜

🌸 Input:

#fsdb date  epoch count
19980201        886320000       6
19980202        886406400       8
19980203        886492800       19
19980204        886579200       53
19980205        886665600       20
19980206        886752000       18
19980207        886838400       5
19980208        886924800       9
19980209        887011200       22
19980210        887097600       22
19980211        887184000       36
19980212        887270400       26
19980213        887356800       23
19980214        887443200       6

🦋 Command:

cat data.fsdb | dbmovingstats -e - -w 4 count
🙈 ok yes the command says dbmovingstats and not dbcolmovingstats — that's literally what the original page says, so we're keeping it exactly. not me being a good archivist 💅

✨ Output:

    #fsdb date epoch count moving_mean moving_stddev
    19980201        886320000       6       -       -
    19980202        886406400       8       -       -
    19980203        886492800       19      -       -
    19980204        886579200       53      21.5    21.764
    19980205        886665600       20      25      19.442
    19980206        886752000       18      27.5    17.02
    19980207        886838400       5       24      20.445
    19980208        886924800       9       13      7.1647
    19980209        887011200       22      13.5    7.8528
    19980210        887097600       22      14.5    8.8129
    19980211        887184000       36      22.25   11.026
    19980212        887270400       26      26.5    6.6081
    19980213        887356800       23      26.75   6.3966
    19980214        887443200       6       22.75   12.473
    #   | dbcolmovingstats -e - -n 4 count

🎀 SEE ALSO 🎀

Fsdb. dbcolstats. dbmultistats. dbrowdiff. go read those too, bestie, they're all part of the friend group 🫶

💀 BUGS 💀

Currently there is no support for generalized quantiles. that's the whole bug section. short king 👑

Copyright (C) 1991-2022 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details.

and that's the tea ☕ — this page is from perl v5.32.1, dated 2024-08-02. periodt 💅✨

📦 where this page comes from