math::statistics(3tcl)

📦
version 2.0 · 2 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

🎀 math::statistics — Basic statistical functions and procedures 🎀

okay bestie, buckle UP, this is math::statistics, part of the Tcl Math Library, and it's about to be your whole personality 💖

✨ SYNOPSIS ✨

package require Tcl 8.5 9

package require math::statistics 1.6.1

::math::statistics::mean data

::math::statistics::min data

::math::statistics::max data

::math::statistics::number data

::math::statistics::stdev data

::math::statistics::var data

::math::statistics::pstdev data

::math::statistics::pvar data

::math::statistics::median data

::math::statistics::basic-stats data

::math::statistics::histogram limits values ?weights?

::math::statistics::histogram-alt limits values ?weights?

::math::statistics::corr data1 data2

::math::statistics::interval-mean-stdev data confidence

::math::statistics::t-test-mean data est_mean est_stdev alpha

::math::statistics::test-normal data significance

::math::statistics::lillieforsFit data

::math::statistics::test-Duckworth list1 list2 significance

::math::statistics::test-anova-F alpha args

::math::statistics::test-Tukey-range alpha args

::math::statistics::test-Dunnett alpha control args

::math::statistics::quantiles data confidence

::math::statistics::quantiles limits counts confidence

::math::statistics::autocorr data

::math::statistics::crosscorr data1 data2

::math::statistics::mean-histogram-limits mean stdev number

::math::statistics::minmax-histogram-limits min max number

::math::statistics::linear-model xdata ydata intercept

::math::statistics::linear-residuals xdata ydata intercept

::math::statistics::test-2x2 n11 n21 n12 n22

::math::statistics::print-2x2 n11 n21 n12 n22

::math::statistics::control-xbar data ?nsamples?

::math::statistics::control-Rchart data ?nsamples?

::math::statistics::test-xbar control data

::math::statistics::test-Rchart control data

::math::statistics::test-Kruskal-Wallis confidence args

::math::statistics::analyse-Kruskal-Wallis args

::math::statistics::test-Levene groups

::math::statistics::test-Brown-Forsythe groups

::math::statistics::group-rank args

::math::statistics::test-Wilcoxon sample_a sample_b

::math::statistics::spearman-rank sample_a sample_b

::math::statistics::spearman-rank-extended sample_a sample_b

::math::statistics::kernel-density data opt -option value ...

::math::statistics::bootstrap data sampleSize ?numberSamples?

::math::statistics::wasserstein-distance prob1 prob2

::math::statistics::kl-divergence prob1 prob2

::math::statistics::logistic-model xdata ydata

::math::statistics::logistic-probability coeffs x

::math::statistics::tstat dof ?alpha?

::math::statistics::mv-wls weights_and_values

::math::statistics::mv-ols values

::math::statistics::pdf-normal mean stdev value

::math::statistics::pdf-lognormal mean stdev value

::math::statistics::pdf-exponential mean value

::math::statistics::pdf-uniform xmin xmax value

::math::statistics::pdf-triangular xmin xmax value

::math::statistics::pdf-symmetric-triangular xmin xmax value

::math::statistics::pdf-gamma alpha beta value

::math::statistics::pdf-poisson mu k

::math::statistics::pdf-chisquare df value

::math::statistics::pdf-student-t df value

::math::statistics::pdf-gamma a b value

::math::statistics::pdf-beta a b value

::math::statistics::pdf-weibull scale shape value

::math::statistics::pdf-gumbel location scale value

::math::statistics::pdf-pareto scale shape value

::math::statistics::pdf-cauchy location scale value

::math::statistics::pdf-laplace location scale value

::math::statistics::pdf-kumaraswamy a b value

::math::statistics::pdf-negative-binomial r p value

::math::statistics::cdf-normal mean stdev value

::math::statistics::cdf-lognormal mean stdev value

::math::statistics::cdf-exponential mean value

::math::statistics::cdf-uniform xmin xmax value

::math::statistics::cdf-triangular xmin xmax value

::math::statistics::cdf-symmetric-triangular xmin xmax value

::math::statistics::cdf-students-t degrees value

::math::statistics::cdf-gamma alpha beta value

::math::statistics::cdf-poisson mu k

::math::statistics::cdf-beta a b value

::math::statistics::cdf-weibull scale shape value

::math::statistics::cdf-gumbel location scale value

::math::statistics::cdf-pareto scale shape value

::math::statistics::cdf-cauchy location scale value

::math::statistics::cdf-F nf1 nf2 value

::math::statistics::cdf-laplace location scale value

::math::statistics::cdf-kumaraswamy a b value

::math::statistics::cdf-negative-binomial r p value

::math::statistics::empirical-distribution values

::math::statistics::random-normal mean stdev number

::math::statistics::random-lognormal mean stdev number

::math::statistics::random-exponential mean number

::math::statistics::random-uniform xmin xmax number

::math::statistics::random-triangular xmin xmax number

::math::statistics::random-symmetric-triangular xmin xmax number

::math::statistics::random-gamma alpha beta number

::math::statistics::random-poisson mu number

::math::statistics::random-chisquare df number

::math::statistics::random-student-t df number

::math::statistics::random-beta a b number

::math::statistics::random-weibull scale shape number

::math::statistics::random-gumbel location scale number

::math::statistics::random-pareto scale shape number

::math::statistics::random-cauchy location scale number

::math::statistics::random-laplace location scale number

::math::statistics::random-kumaraswamy a b number

::math::statistics::random-negative-binomial r p number

::math::statistics::histogram-uniform xmin xmax limits number

::math::statistics::incompleteGamma x p ?tol?

::math::statistics::incompleteBeta a b x ?tol?

::math::statistics::estimate-pareto values

::math::statistics::estimate-exponential values

::math::statistics::estimate-laplace values

::math::statistics::estimante-negative-binomial r values

::math::statistics::filter varname data expression

::math::statistics::map varname data expression

::math::statistics::samplescount varname list expression

::math::statistics::subdivide

::math::statistics::plot-scale canvas xmin xmax ymin ymax

::math::statistics::plot-xydata canvas xdata ydata tag

::math::statistics::plot-xyline canvas xdata ydata tag

::math::statistics::plot-tdata canvas tdata tag

::math::statistics::plot-tline canvas tdata tag

::math::statistics::plot-histogram canvas counts limits tag

🌸 DESCRIPTION 🌸

So the math::statistics package is literally a whole toolbox of functions and procedures for basic statistical data analysis, like:

  • Descriptive statistical parameters (mean, minimum, maximum, standard deviation) 📊
  • Estimates of the distribution in the form of histograms and quantiles 🎀
  • Basic testing of hypotheses 🧪
  • Probability and cumulative density functions 🌈

It's here to help you build data analysis applications or just do some ad hoc data analysis on the fly. It is not in itself a full application, and no cap, it is not trying to rival full (non-)commercial statistical packages. It's giving helper, not competitor 💅

The purpose of this document is to describe the implemented procedures and give you some examples of their usage. Since there's ample literature on the algorithms involved, we just point you at the relevant text books for more explanation, iykyk 📚

The package contains a fairly large number of public procedures. They split into three sets: general procedures, procedures that deal with specific statistical distributions, list procedures to select or transform data, and simple plotting procedures (these require Tk).

Note: the data that needs to be analyzed is always contained in a simple list. Missing values are represented as empty list elements. 🫶
Note: with version 1.0.1 a mistake in the procs pdf-lognormal, cdf-lognormal and random-lognormal has been corrected. In previous versions the argument for the standard deviation was actually used as if it was the variance. 😭 (glad that's fixed fr fr)

👑 GENERAL PROCEDURES 👑

The general statistical procedures are:

  • ::math::statistics::mean data — Determine the mean value of the given list of data. 🍓
    • list data — List of data
  • ::math::statistics::min data — Determine the minimum value of the given list of data.
    • list data — List of data
  • ::math::statistics::max data — Determine the maximum value of the given list of data.
    • list data — List of data
  • ::math::statistics::number data — Determine the number of non-missing data in the given list. ✨
    • list data — List of data
  • ::math::statistics::stdev data — Determine the sample standard deviation of the data in the given list.
    • list data — List of data
  • ::math::statistics::var data — Determine the sample variance of the data in the given list.
    • list data — List of data
  • ::math::statistics::pstdev data — Determine the population standard deviation of the data in the given list.
    • list data — List of data
  • ::math::statistics::pvar data — Determine the population variance of the data in the given list.
    • list data — List of data
  • ::math::statistics::median data — Determine the median of the data in the given list. (Note that this requires sorting the data, which may be a costly operation 💀)
    • list data — List of data
  • ::math::statistics::basic-stats data — Determine a list of all the descriptive parameters: mean, minimum, maximum, number of data, sample standard deviation, sample variance, population standard deviation and population variance. 🎀

(This routine is called whenever either or all of the basic statistical parameters are required. Hence all calculations are done and the relevant values are returned.)

  • list data — List of data
  • ::math::statistics::histogram limits values ?weights? — Determine histogram information for the given list of data. Returns a list consisting of the number of values that fall into each interval. (The first interval consists of all values lower than the first limit, the last interval consists of all values greater than the last limit. There is one more interval than there are limits.) 📊

Optionally, you can use weights to influence the histogram.

  • limits (list) — the upper limits for the histogram intervals, in ascending order ⬆️
  • values (list) — the data itself, the whole reason we're here 💖
  • weights (list) — one weight per value, no more no less 🎀

🦋 ::math::statistics::histogram-alt limits values ?weights?

Okay so this is literally the alt version of the histogram proc — same energy, different edge. The only glow-up: the open end of the intervals sits at the lower bound instead of the upper bound. That's it. That's the whole difference. 🔥

  • limits (list) — upper limits (ascending, bestie, ascending) for the intervals of the histogram
  • values (list) — list of data
  • weights (list) — list of weights, one weight per value

💫 ::math::statistics::corr data1 data2

Find the correlation coefficient between two sets of data. Two datasets walk in, one number walks out. ✨

  • data1 (list) — first list of data
  • data2 (list) — second list of data

🎀 ::math::statistics::interval-mean-stdev data confidence

Hands you the interval containing the mean value, plus one containing the standard deviation, both at a certain level of confidence. Heads up: this assumes a normal distribution, so don't come for it if your data is chaotic 🙈

  • data (list) — list of raw data values (small sample)
  • confidence (float) — confidence level (like 0.95 or 0.99 for instance)

🌸 ::math::statistics::t-test-mean data est_mean est_stdev alpha

Checks whether the mean value of a sample lines up with the estimated normal distribution, at a certain probability. Returns 1 if the test succeeds, or 0 if the mean is unlikely to fit the distribution you gave it. Basically a vibe check for your mean 👑

  • data (list) — list of raw data values (small sample)
  • est_mean (float) — estimated mean of the distribution
  • est_stdev (float) — estimated stdev of the distribution
  • alpha (float) — probability level (like 0.95 or 0.99 for instance)

🧚 ::math::statistics::test-normal data significance

Does this data follow a normal distribution at a given level of significance? Returns 1 if the data are normally distributed within that level of significance, and 0 if not. The test underneath is the Lilliefors test. And lowkey important: smaller values of the significance mean a stricter testing. 😤

  • data (list) — list of raw data values
  • significance (float) — significance level (one of 0.01, 0.05, 0.10, 0.15 or 0.20). For compatibility reasons the values "1-significance", 0.80, 0.85, 0.90, 0.95 or 0.99 are also accepted. iykyk 🫶
Compatibility issue: the original implementation and documentation used the term "confidence" and used a value 1-significance (see ticket 2812473fff). This has been corrected as of version 0.9.3. 💀

🌈 ::math::statistics::lillieforsFit data

Returns the goodness of fit to a normal distribution according to Lilliefors. The higher the number, the more likely the data really are normally distributed. And bestie, this test requires at least five data points — five or nothing, periodt. ✋

  • data (list) — list of raw data values

💜 ::math::statistics::test-Duckworth list1 list2 significance

Do two data sets have the same median? This is the Tukey-Duckworth test deciding that for you.

Returns:

  • 0 if the medians are unequal
  • 1 if they are equal
  • -1 if the test can not be conducted — that happens when the smallest value must be in a different set than the greatest value 😭
And yeah, the source page literally has a chunk of comment-looking text pasted in right here, so I'm giving it to you exactly as it shows up, no edits, no cap:
# # Arguments: #     list1           Values in the first
data set #     list2           Values in the second data set #
significance    Significance level (either 0.05, 0.01 or 0.001) #
# Returns: Test whether the given data follow a normal
distribution with a certain level of significance.  Returns 1 if
the data are normally distributed within the level of
significance, returns 0 if not. The underlying test is the
Lilliefors test. Smaller values of the significance mean a
stricter testing.
  • list1 (list) — first list of data
  • list2 (list) — second list of data
  • significance (float) — significance level (either 0.05, 0.01 or 0.001)

⭐ ::math::statistics::test-anova-F alpha args

Do two or more groups of normally distributed data have the same means? This is a one-way ANOVA test. It returns 0 if the means are likely unequal, 1 if they are. 💅

It also returns a list of the comparison results for each pair of groups. Each element of that list contains: the index of the first group and that of the second group, whether the means are likely to be different (1) or not (0), and the confidence interval the conclusion is based on. The groups may also be stored in a nested list — so both of these are the same thing:

test-anova-F 0.05 $A $B $C
#
# Or equivalently:
#
test-anova-F 0.05 [list $A $B $C]
  • alpha (float) — significance level
  • args (list) — two or more groups of data to be checked

🔥 ::math::statistics::test-Tukey-range alpha args

Same question — do two or more groups of normally distributed data have the same means — but this time via Tukey's range test. It's complementary to the ANOVA test, so they're a duo, not a rivalry 🫶

The proc returns a list of the comparison results for each pair of groups. Each element contains: the index of the first group and that of the second group, whether the means are likely to be different (1) or not (0), and the confidence interval the conclusion is based on. Groups can also be stored in a nested list, just like with the ANOVA test.

  • alpha (float) — significance level — either 0.05 or 0.01
  • args (list) — two or more groups of data to be checked

🎀 ::math::statistics::test-Dunnett alpha control args

Do one or more groups of normally distributed data have the same means as the group of control data? That's Dunnett's test, and it's complementary to the ANOVA test too.

The proc returns a list of the comparison results for each group against the control group. Each element of this list contains: whether the means are likely to be different (1) or not (0), and the confidence interval the conclusion is based on. Groups may also be stored in a nested list, just as with the ANOVA test. ✨

Note: some care is required if there is only one group to compare the control with: test-Dunnett-F 0.05 $control [list $A] Otherwise group A is split up into groups of one element — this is due to an ambiguity. 🙈
  • alpha (float) — significance level — either 0.05 or 0.01
  • args (list) — one or more groups of data to be checked

🍓 ::math::statistics::quantiles data confidence

Returns the quantiles for a given set of data. Slay. ✨

  • data (list) — list of raw data values
  • confidence (float) — confidence level (like 0.95 or 0.99 for instance) or a list of confidence levels

🌸 ::math::statistics::quantiles limits counts confidence

Returns the quantiles based on histogram information — this is the alternative to the call with two arguments. Same name, different energy 👑

  • limits (list) — list of upper limits from histogram
  • counts (list) — list of counts for each interval in histogram
  • confidence (float) — confidence level (like 0.95 or 0.99 for instance) or a list of confidence levels

🦋 ::math::statistics::autocorr data

Returns the autocorrelation function as a list of values (assuming equidistance between samples, about 1/2 of the number of raw data). 💫

The correlation is determined in such a way that the first value is always 1 and all others are equal to or smaller than 1. The number of values involved will diminish as the "time" (the index in the list of returned values) increases.

  • data (list) — raw data for which the autocorrelation must be determined

💜 ::math::statistics::crosscorr data1 data2

Returns the cross-correlation function as a list of values (assuming equidistance between samples, about 1/2 of the number of raw data). 🌈

The correlation is determined in such a way that the values can never exceed 1 in magnitude. The number of values involved will diminish as the "time" (the index in the list of returned values) increases.

  • data1 (list) — first list of data
  • data2 (list) — second list of data

💖 ::math::statistics::mean-histogram-limits mean stdev number

Works out reasonable limits based on mean and standard deviation for a histogram. It's a convenience function, bestie — the result is suitable for the histogram function. 🎀

  • mean (float) — mean of the data
  • stdev (float) — standard deviation
  • number (int) — number of limits to generate (defaults to 8)

⭐ ::math::statistics::minmax-histogram-limits min max number

Works out reasonable limits based on a minimum and a maximum for a histogram. Another convenience function — the result is suitable for the histogram function. 🔥

  • min (float) — expected minimum
  • max (float) — expected maximum
  • number (int) — number of limits to generate (defaults to 8)

👑 ::math::statistics::linear-model xdata ydata intercept

Finds the coefficients for a linear regression between two series of data — the model being Y = A + B*X. Returns a list of parameters describing the fit. Main character energy fr 💅

  • xdata (list) — list of independent data
  • ydata (list) — list of dependent data to be fitted
  • intercept (boolean) — (optional) compute the intercept (1, the default) or fit to a line through the origin (0)

The result consists of the following list:

  • (Estimate of) Intercept A
  • (Estimate of) Slope B
  • Standard deviation of Y relative to fit
  • Correlation coefficient R2 💖
  • Number of degrees of freedom df
  • Standard error of the intercept A
  • Significance level of A
  • Standard error of the slope B
  • Significance level of B

🌈 ::math::statistics::linear-residuals xdata ydata intercept

Okay so this one is giving "how far off was our little line, actually?" energy ✨ It figures out the difference between your real data and whatever the linear model predicted.

Returns a list of the differences between the actual data and the predicted values. 💅

  • list xdata — List of independent data
  • list ydata — List of dependent data to be fitted
  • boolean intercept — (Optional) compute the intercept (1, default) or fit to a line through the origin (0)

🦋 ::math::statistics::test-2x2 n11 n21 n12 n22

Two sets of samples, both from a binomial distribution — do they differ significantly or not (which would imply a different parameter)? 👀 That's the whole vibe.

Returns the "chi-square" value, which you can use to determine the significance. 🔥

  • int n11 — Number of outcomes with the first value from the first sample.
  • int n21 — Number of outcomes with the first value from the second sample.
  • int n12 — Number of outcomes with the second value from the first sample.
  • int n22 — Number of outcomes with the second value from the second sample.

🎀 ::math::statistics::print-2x2 n11 n21 n12 n22

Same question as above, bestie — two sets of samples, each from a binomial distribution, do they differ significantly or not (implying a different parameter)? 🫶

Returns a short report, useful in an interactive session. Like, you're poking at numbers in a live shell and you just want the tea. ☕

  • int n11 — Number of outcomes with the first value from the first sample.
  • int n21 — Number of outcomes with the first value from the second sample.
  • int n12 — Number of outcomes with the second value from the first sample.
  • int n22 — Number of outcomes with the second value from the second sample.

👑 ::math::statistics::control-xbar data ?nsamples?

Control limits for an xbar chart, let's gooo ✨ The number of data in each subsample defaults to 4. And heads up: at least 20 subsamples are required, that's the rule, no cap.

Returns the mean, the lower limit, the upper limit and the number of data per subsample. 💅

  • list data — List of observed data
  • int nsamples — Number of data per subsample

📊 ::math::statistics::control-Rchart data ?nsamples?

Control limits for an R chart this time 🌈 The number of data in each subsample (nsamples) defaults to 4. Same deal as above: at least 20 subsamples are required.

Returns the mean range, the lower limit, the upper limit and the number of data per subsample. 🎀

  • list data — List of observed data
  • int nsamples — Number of data per subsample

🔍 ::math::statistics::test-xbar control data

Did the data actually go and exceed the control limits for the xbar chart? Rude of it, honestly. 😭

Returns a list of subsamples (their indices) that indeed violate the limits. So like, the naughty list. 💀

  • list control — Control limits as returned by the "control-xbar" procedure
  • list data — List of observed data

🔍 ::math::statistics::test-Rchart control data

Same energy, R chart edition ✨ Did the data exceed the control limits for the R chart?

Returns a list of subsamples (their indices) that indeed violate the limits. 💅

  • list control — Control limits as returned by the "control-Rchart" procedure
  • list data — List of observed data

🌸 ::math::statistics::test-Kruskal-Wallis confidence args

Are the population medians of two or more groups equal, at a given confidence level? Kruskal-Wallis test says let's find out 🫶

  • float confidence — Confidence level to be used (0-1)
  • list args — Two or more lists of data

💜 ::math::statistics::analyse-Kruskal-Wallis args

Crunch the statistical parameters for the Kruskal-Wallis test. Returns the Kruskal-Wallis statistic and the probability that that value would occur assuming the medians of the populations are equal. 👑

  • list args — Two or more lists of data

🧚 ::math::statistics::test-Levene groups

Compute the Levene statistic to determine if groups of data have the same variance (are homoscadastic) or not. The data are organised in groups. This version uses the mean of the data as the measure to determine the deviations. The statistic is equivalent to an F statistic with degrees of freedom k-1 and N-k, k being the number of groups and N the total number of data. 🔥

  • list groups — List of groups of data

🧚 ::math::statistics::test-Brown-Forsythe groups

Compute the Brown-Forsythe statistic to determine if groups of data have the same variance (are homoscadastic) or not. Like the Levene test, but this version uses the median of the data. Sibling energy fr fr 💖

  • list groups — List of groups of data

⭐ ::math::statistics::group-rank args

Rank the groups of data with respect to the complete set. Returns a list consisting of the group ID, the value and the rank (possibly a rational number, in case of ties) for each data item. Ties get the fractional treatment, it's giving fairness 💅

  • list args — Two or more lists of data

🌈 ::math::statistics::test-Wilcoxon sample_a sample_b

Compute the Wilcoxon test statistic to determine if two samples have the same median or not. (The statistic can be regarded as standard normal, if the sample sizes are both larger than 10.) Returns the value of this statistic. ✨

  • list sample_a — List of data comprising the first sample
  • list sample_b — List of data comprising the second sample

🦋 ::math::statistics::spearman-rank sample_a sample_b

Return the Spearman rank correlation as an alternative to the ordinary (Pearson's) correlation coefficient. The two samples should have the same number of data — like, matching lengths, bestie, that's the deal. 🫶

  • list sample_a — First list of data
  • list sample_b — Second list of data

👑 ::math::statistics::spearman-rank-extended sample_a sample_b

Spearman rank correlation as an alternative to the ordinary (Pearson's) correlation coefficient as well as additional data 🌟 The two samples should have the same number of data. The procedure returns the correlation coefficient, the number of data pairs used and the z-score, an approximately standard normal statistic, indicating the significance of the correlation. So much main character information. 💅

  • list sample_a — First list of data
  • list sample_b — Second list of data

🎀 ::math::statistics::kernel-density data opt -option value ...

Return the density function based on kernel density estimation. The procedure is controlled by a small set of options, each of which is given a reasonable default. ✨

The return value consists of three lists: the centres of the bins, the associated probability density and a list of computational parameters (begin and end of the interval, mean and standard deviation and the used bandwidth). The computational parameters can be used for further analysis. Obsessed with a return value that keeps receipts. 🧾💖

  • list data — The data to be examined
  • list args — Option-value pairs:
    • -weights weights — Per data point the weight (default: 1 for all data)
    • -bandwidth value — Bandwidth to be used for the estimation (default: determined from standard deviation)
    • -number value — Number of bins to be returned (default: 100)
    • -interval {begin end} — Begin and end of the interval for which the density is returned (default: mean +/- 3*standard deviation)
    • -kernel function — Kernel to be used (One of: gaussian, cosine, epanechnikov, uniform, triangular, biweight, logistic; default: gaussian)

🍓 ::math::statistics::bootstrap data sampleSize ?numberSamples?

Create a subsample or subsamples from a given list of data 🎀 The data in the samples are chosen from this list — multiples may occur, that's not a bug, that's the whole point. If there is only one subsample, the sample itself is returned (as a list of "sampleSize" values), otherwise a list of samples is returned. 💅

  • list data — List of values to chose from
  • int sampleSize — Number of values per sample
  • int numberSamples — Number of samples (default: 1)

🌸 ::math::statistics::wasserstein-distance prob1 prob2

Compute the Wasserstein distance or earth mover's distance for two equidstantly spaced histograms or probability densities. The histograms need not to be normalised to sum to one, but they must have the same number of entries. Same length or it's not happening, bestie. 😤

Note: the histograms are assumed to be based on the same equidistant intervals. As the bounds are not passed, the value is expressed in the length of the intervals.
  • list prob1 — List of values for the first histogram/probability density
  • list prob2 — List of values for the second histogram/probability density

💜 ::math::statistics::kl-divergence prob1 prob2

Compute the Kullback-Leibler (KL) divergence for two equidstantly spaced histograms or probability densities. The histograms need not to be normalised to sum to one, but they must have the same number of entries. ✨

Note: the histograms are assumed to be based on the same equidistant intervals. As the bounds are not passed, the value is expressed in the length of the intervals.
  • list prob1 — List of values for the first histogram/probability density
  • list prob2 — List of values for the second histogram/probability density

Alsooo, quick heads up on KL divergence: it's not symmetric, and hist2 should not have zeroes anywhere hist1 has non-zero values. no cap, that matters 💀

  • list prob1 — the list of values for the first histogram / probability density 🫶
  • list prob2 — the list of values for the second histogram / probability density 🫶
  • ::math::statistics::logistic-model xdata ydata — fits the logistic model to your data, bestie. The data is independent x-values plus an outcome of 0 or 1 for each one, and the result lets you estimate the probability that some x-value gives you a 1. ✨
    • list xdata — the values where you already know the success (1) or failure (0)
    • list ydata — the successes or failures that go with each value in xdata.
  • ::math::statistics::logistic-probability coeffs x — gives you the probability of success for the value x, once you hand it the logistic model's coefficients. 💅
    • list coeffs — the coefficient list, as produced by the logistic-model command
    • float x — the x-value you want the probability for

👑 MULTIVARIATE LINEAR REGRESSION 👑

Okay so beyond plain linear regression with a single independent variable, this package also does ordinary least squares (OLS) and weighted least squares (WLS) linear regression with several variables. Two whole procedures for it, and they were written by Eric Kemp-Benedict — iconic. 🫶

On top of those two, there's also a procedure (tstat) that computes the t-statistic value for a given number of degrees of freedom, the one you need to show a given level of significance.

Heads up bestie: these procedures depend on the math::linearalgebra package. don't forget it ⭐

🌸 Description of the procedures 🌸

  • ::math::statistics::tstat dof ?alpha? — returns the value of the t-distribution t* satisfying
P(t*)  =  1 - alpha/2
P(-t*) =  alpha/2

for the number of degrees of freedom dof.

So if you've got a sample of normally-distributed data x, with an estimate xbar for the mean and sbar for the standard deviation, the alpha confidence interval for the mean estimate works out to

  ( xbar - t* sbar , xbar + t* sbar)

And what you do with the return value? You compare it against an estimated t-statistic to figure out whether a parameter's estimated value is significantly different from zero at that confidence level. 🎀

  • int dof — number of degrees of freedom
  • float alpha — confidence level of the t-distribution. Defaults to 0.05.
  • ::math::statistics::mv-wls weights_and_values — runs a weighted least squares linear regression on your data points, with a weight for each point. 💅

The linear model is of the form

  y = b0 + b1 * x1 + b2 * x2 ... + bN * xN + error

and every point satisfies

  yi = b0 + b1 * xi1 + b2 * xi2 + ... + bN * xiN + Residual_i

The procedure returns a list with these elements:

  • The r-squared statistic
  • The adjusted r-squared statistic
  • A list containing the estimated coefficients b1, ... bN, b0 — and yes, the constant b0 comes last in that list, don't @ me 🙈
  • A list containing the standard errors of the coefficients
  • A list containing the 95% confidence bounds of the coefficients, with each set of bounds returned as a list with two values

Arguments:

  • list weights_and_values — a list consisting of: the weight for the first observation, the data for the first observation (as a sublist), the weight for the second observation (as a sublist), and so on. Those data sublists are organised as lists of the value of the dependent variable y followed by the independent variables x1, x2 up to xN.

Example of the use: the weight factors here are super simple — 0.2 for negative values to show we trust those observations less, and 1.0 for all positive values. ✨

# Store the value of the unicode value for the "+/-" character
set pm "\u00B1"

# Provide some data
set data { 0.2 {  -.67  14.18  60.03 -7.5  }
           1.0 { 36.97  15.52  34.24 14.61 }
           0.2 {-29.57  21.85  83.36 -7.   }
           0.2 {-16.9   11.79  51.67 -6.56 }
           1.0 { 14.09  16.24  36.97 -12.84}
           1.0 { 31.52  20.93  45.99 -25.4 }
           1.0 { 24.05  20.69  50.27  17.27}
           1.0 { 22.23  16.91  45.07  -4.3 }
           1.0 { 40.79  20.49  38.92  -.73 }
           0.2 {-10.35  17.24  58.77  18.78}}

# Call the ols routine
set results [::math::statistics::mv-ols $data]

# Pretty-print the results
puts "R-squared: [lindex $results 0]"
puts "Adj R-squared: [lindex $results 1]"
puts "Coefficients $pm s.e. -- \[95% confidence interval\]:"
foreach val [lindex $results 2] se [lindex $results 3] bounds [lindex $results 4] {
    set lb [lindex $bounds 0]
    set ub [lindex $bounds 1]
    puts "   $val $pm $se -- \[$lb to $ub\]"
}
  • ::math::statistics::mv-ols values — runs an ordinary least squares linear regression on your data points. 🦋

This one is lowkey just calling ::mvlinreg::wls with all weights set to 1.0, and it hands back the exact same information.

Example of the use:

# Store the value of the unicode value for the "+/-" character
set pm "\u00B1"

# Provide some data
set data {{  -.67  14.18  60.03 -7.5  }
          { 36.97  15.52  34.24 14.61 }
          {-29.57  21.85  83.36 -7.   }
          {-16.9   11.79  51.67 -6.56 }
          { 14.09  16.24  36.97 -12.84}
          { 31.52  20.93  45.99 -25.4 }
          { 24.05  20.69  50.27  17.27}
          { 22.23  16.91  45.07  -4.3 }
          { 40.79  20.49  38.92  -.73 }
          {-10.35  17.24  58.77  18.78}}

# Call the ols routine
set results [::math::statistics::mv-ols $data]

# Pretty-print the results
puts "R-squared: [lindex $results 0]"
puts "Adj R-squared: [lindex $results 1]"
puts "Coefficients $pm s.e. -- \[95% confidence interval\]:"
foreach val [lindex $results 2] se [lindex $results 3] bounds [lindex $results 4] {
    set lb [lindex $bounds 0]
    set ub [lindex $bounds 1]
    puts "   $val $pm $se -- \[$lb to $ub\]"
}

🌈 STATISTICAL DISTRIBUTIONS 🌈

The literature has so many probability distributions, it's giving infinity gauntlet honestly. This package supports:

  • The normal or Gaussian distribution, plus the log-normal distribution 💜
  • The uniform distribution — equal probability for all data inside a given interval
  • The exponential distribution — useful as a model for certain extreme-value distributions
  • The gamma distribution — based on the incomplete Gamma integral
  • The beta distribution
  • The chi-square distribution
  • The student's T distribution
  • The Poisson distribution
  • The Pareto distribution
  • The Gumbel distribution
  • The Weibull distribution
  • The Cauchy distribution
  • The F distribution (only the cumulative density function) 🙈
  • PM - binomial.

In principle, for each distribution you get procedures for:

  • The probability density (pdf-*) ✨
  • The cumulative density (cdf-*) ✨
  • Quantiles for the given distribution (quantiles-*) ✨
  • Histograms for the given distribution (histogram-*) ✨
  • A list of random values with the given distribution (random-*) ✨

The following procedures have been implemented:

  • ::math::statistics::pdf-normal mean stdev value — returns the probability of a given value for a normal distribution with the given mean and standard deviation. 🎀
    • float mean — mean value of the distribution
    • float stdev — standard deviation of the distribution
    • float value — value for which the probability is required
  • ::math::statistics::pdf-lognormal mean stdev value — returns the probability of a given value for a log-normal distribution with the given mean and standard deviation. 🌸
    • float mean — mean value of the distribution
    • float stdev — standard deviation of the distribution
    • float value — value for which the probability is required
  • ::math::statistics::pdf-exponential mean value — returns the probability of a given value for an exponential distribution with the given mean. 💅
    • float mean — mean value of the distribution
    • float value — value for which the probability is required
  • ::math::statistics::pdf-uniform xmin xmax value — returns the probability of a given value for a uniform distribution with the given extremes. 🦋
    • float xmin — minimum value of the distribution
    • float xmin — maximum value of the distribution
    • float value — value for which the probability is required
  • ::math::statistics::pdf-triangular xmin xmax value — returns the probability of a given value for a triangular distribution with the given extremes. And here's the cute part: if the argument min is lower than the argument max, then smaller values have higher probability, and vice versa. In the first case the probability density function is of the form f(x) = 2(1-x), and in the other case it's of the form f(x) = 2x. 😌
    • float xmin — minimum value of the distribution
    • float xmin — maximum value of the distribution
    • float value — value for which the probability is required
  • ::math::statistics::pdf-symmetric-triangular xmin xmax value — returns the probability of a given value for a symmetric triangular distribution with the given extremes. 👑
    • float xmin — minimum value of the distribution

(…aaand finishing off the previous entry's params real quick 💅)

  • float xmin — the maximum value of the distribution
  • float value — the value you want the probability for
  • ::math::statistics::pdf-gamma alpha beta value — hands you the probability of a given value for a Gamma distribution when you feed it shape and rate params.
    • float alpha — shape parameter, yes ma'am
    • float beta — rate parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-poisson mu k — gives you the probability of a given number of occurrences in the same interval (k) for a Poisson distribution with the mean you gave it (mu). so iconic.
    • float mu — the mean number of occurrences
    • int k — number of occurences (spelling is the original's, not mine, don't come for me 🙈)
  • ::math::statistics::pdf-chisquare df value — probability of a given value for a chi square distribution with the degrees of freedom you specify.
    • float df — degrees of freedom
    • float value — the value you want the probability for
  • ::math::statistics::pdf-student-t df value — probability of a given value for a Student's t distribution with the given degrees of freedom.
    • float df — degrees of freedom
    • float value — the value you want the probability for
  • ::math::statistics::pdf-gamma a b value — yeah it's another one bestie ✨ returns the probability of a given value for a Gamma distribution with given shape and rate parameters.
    • float a — shape parameter
    • float b — rate parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-beta a b value — probability of a given value for a Beta distribution with the shape parameters you hand it.
    • float a — first shape parameter
    • float b — second shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-weibull scale shape value — probability of a given value for a Weibull distribution with given scale and shape parameters.
    • float location — scale parameter
    • float scale — shape parameter
    • float value — the value you want the probability for
okay genuine heads-up: that param list is exactly what the original says (a location-named param described as the scale, and a scale-named param described as the shape). not me editing it. copy it as-is fr fr 💀
  • ::math::statistics::pdf-gumbel location scale value — probability of a given value for a Gumbel distribution with given location and shape parameters.
    • float location — location parameter
    • float scale — shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-pareto scale shape value — probability of a given value for a Pareto distribution with given scale and shape parameters.
    • float scale — scale parameter
    • float shape — shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-cauchy location scale value — probability of a given value for a Cauchy distribution with given location and shape parameters. and bestie, lowkey important: the Cauchy distribution has no finite higher-order moments. none. zero. it simply does not have them. 🫠
    • float location — location parameter
    • float scale — shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-laplace location scale value — probability of a given value for a Laplace distribution with given location and shape parameters. the Laplace distribution is built from two exponential functions, it's peaked, and its tails are heavier than the normal distribution's. main character distribution energy ngl 👑
    • float location — location parameter (the mean)
    • float scale — shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::pdf-kumaraswamy a b value — probability of a given value for a Kumaraswamy distribution with parameters a and b. it's related to the Beta distribution, but its cumulative distribution function is tractable. that's the whole flex 💅
    • float a — parameter a
    • float b — parameter b
    • float value — the value you want the probability for
  • ::math::statistics::pdf-negative-binomial r p value — probability of a given value for a negative binomial distribution, with an allowed number of failures and the probability of success.
    • int r — allowed number of failures (at least 1, so don't try 0 babe)
    • float p — probability of success
    • int value — number of successes you want the probability for
  • ::math::statistics::cdf-normal mean stdev value — the cumulative probability of a given value for a normal distribution with the mean and standard deviation you gave it. that's the probability for values up to the given one, so everything to the left. slay.
    • float mean — mean value of the distribution
    • float stdev — standard deviation of the distribution
    • float value — the value you want the probability for
  • ::math::statistics::cdf-lognormal mean stdev value — cumulative probability of a given value for a log-normal distribution with the given mean and standard deviation, i.e. the probability for values up to the given one.
    • float mean — mean value of the distribution
    • float stdev — standard deviation of the distribution
    • float value — the value you want the probability for
  • ::math::statistics::cdf-exponential mean value — cumulative probability of a given value for an exponential distribution with the given mean.
    • float mean — mean value of the distribution
    • float value — the value you want the probability for
  • ::math::statistics::cdf-uniform xmin xmax value — cumulative probability of a given value for a uniform distribution with the given extremes.
    • float xmin — minimum value of the distribution
    • float xmin — maximum value of the distribution
    • float value — the value you want the probability for
again, that's xmin listed twice in the source. not a typo I made, not a typo I'm fixing. reproduce it exactly 😭
  • ::math::statistics::cdf-triangular xmin xmax value — cumulative probability of a given value for a triangular distribution with the given extremes. heads up: if xmin < xmax, then lower values have a higher probability, and vice versa. see also pdf-triangular.
    • float xmin — minimum value of the distribution
    • float xmin — maximum value of the distribution
    • float value — the value you want the probability for
  • ::math::statistics::cdf-symmetric-triangular xmin xmax value — cumulative probability of a given value for a symmetric triangular distribution with the given extremes. symmetric. balanced. the way she should be ✨
    • float xmin — minimum value of the distribution
    • float xmin — maximum value of the distribution
    • float value — the value you want the probability for
  • ::math::statistics::cdf-students-t degrees value — cumulative probability of a given value for a Student's t distribution with the given number of degrees.
    • int degrees — number of degrees of freedom
    • float value — the value you want the probability for
  • ::math::statistics::cdf-gamma alpha beta value — cumulative probability of a given value for a Gamma distribution with given shape and rate parameters.
    • float alpha — shape parameter
    • float beta — rate parameter
    • float value — the value you want the cumulative probability for
  • ::math::statistics::cdf-poisson mu k — cumulative probability of a given number of occurrences in the same interval (k) for a Poisson distribution with the given mean (mu).
    • float mu — the mean number of occurrences
    • int k — number of occurences
  • ::math::statistics::cdf-beta a b value — cumulative probability of a given value for a Beta distribution with the given shape parameters.
    • float a — first shape parameter
    • float b — second shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::cdf-weibull scale shape value — cumulative probability of a given value for a Weibull distribution with the given scale and shape parameters.
    • float scale — scale parameter
    • float shape — shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::cdf-gumbel location scale value — cumulative probability of a given value for a Gumbel distribution with the given location and scale parameters.
    • float location — location parameter
    • float scale — scale parameter
    • float value — the value you want the probability for
  • ::math::statistics::cdf-pareto scale shape value — cumulative probability of a given value for a Pareto distribution with the given scale and shape parameters.
    • float scale — scale parameter
    • float shape — shape parameter
    • float value — the value you want the probability for
  • ::math::statistics::cdf-cauchy location scale value — cumulative probability of a given value for a Cauchy distribution with the given location and scale parameters.
    • float location — location parameter
    • float scale — scale parameter
  • float value — the value you want the probability for 💅 (yeah, this is the last arg of the cdf procedure we were just vibing about)

🎀 more cdf besties (the distribution glow-up continues) 🎀

  • ::math::statistics::cdf-F nf1 nf2 value — hands you the cumulative probability of a given value for an F distribution with nf1 and nf2 degrees of freedom. fr fr, it's giving "are these two variances actually different".
    • float nf1 — degrees of freedom for the numerator 👑
    • float nf2 — degrees of freedom for the denominator 💜
    • float value — the value you want the probability for
  • ::math::statistics::cdf-laplace location scale value — the cumulative probability of a given value for a Laplace distribution with the given location and shape parameters. bestie, the Laplace distribution is literally two exponential functions stitched together, it's peaked and has heavier tails than the normal distribution. iconic behaviour honestly.
    • float location — location parameter (mean) 📍
    • float scale — shape parameter 🦋
    • float value — the value you want the probability for
  • ::math::statistics::cdf-kumaraswamy a b value — cumulative probability of a given value for a Kumaraswamy distribution with parameters a and b. this one is related to the Beta distribution but has a tractable cumulative distribution function, which is the whole point, no cap. ✨
    • float a — parameter a
    • float b — parameter b
    • float value — the value you want the probability for
  • ::math::statistics::cdf-negative-binomial r p value — cumulative probability of a given value for a negative binomial distribution, given the allowed number of failures and the probability of success. 🍓
    • int r — allowed number of failures (at least 1)
    • float p — probability of success
    • int value — greatest number of successes
  • ::math::statistics::empirical-distribution values — returns a list of values and their empirical probability, sorted in increasing order. (the implementation follows the description at the corresponding Wikipedia page, iykyk 📚)
    • list values — list of data to be examined

🌈 random value generators (the fun part) 🌈

these all spit out a list of number random values from whatever distribution you ask for. yes, plural, a whole list. slay.

  • ::math::statistics::random-normal mean stdev number — returns a list of number random values satisfying a normal distribution with the given mean and standard deviation. the classic. 💖
    • float mean — mean value of the distribution
    • float stdev — standard deviation of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-lognormal mean stdev number — returns a list of number random values satisfying a log-normal distribution with the given mean and standard deviation. 🦋
    • float mean — mean value of the distribution
    • float stdev — standard deviation of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-exponential mean number — returns a list of number random values satisfying an exponential distribution with the given mean. ⭐
    • float mean — mean value of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-uniform xmin xmax number — returns a list of number random values satisfying a uniform distribution with the given extremes. 🌸
    • float xmin — minimum value of the distribution
    • float xmax — maximum value of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-triangular xmin xmax number — returns a list of number random values satisfying a triangular distribution with the given extremes. heads up: if xmin < xmax, then lower values have a higher probability, and vice versa (see also pdf-triangular). 🙈
    • float xmin — minimum value of the distribution
    • float xmax — maximum value of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-symmetric-triangular xmin xmax number — returns a list of number random values satisfying a symmetric triangular distribution with the given extremes. balanced. centered. main character energy. 👑
    • float xmin — minimum value of the distribution
    • float xmax — maximum value of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-gamma alpha beta number — returns a list of number random values satisfying a Gamma distribution with the given shape and rate parameters. 🔥
    • float alpha — shape parameter
    • float beta — rate parameter
    • int number — number of values to be returned
  • ::math::statistics::random-poisson mu number — returns a list of number random values satisfying a Poisson distribution with the given mean. 🍓
    • float mu — mean of the distribution
    • int number — number of values to be returned
  • ::math::statistics::random-chisquare df number — returns a list of number random values satisfying a chi square distribution with the given degrees of freedom. 💜
    • float df — degrees of freedom
    • int number — number of values to be returned
  • ::math::statistics::random-student-t df number — returns a list of number random values satisfying a Student's t distribution with the given degrees of freedom. 🎀
    • float df — degrees of freedom
    • int number — number of values to be returned
  • ::math::statistics::random-beta a b number — returns a list of number random values satisfying a Beta distribution with the given shape parameters. ✨
    • float a — first shape parameter
    • float b — second shape parameter
    • int number — number of values to be returned
  • ::math::statistics::random-weibull scale shape number — returns a list of number random values satisfying a Weibull distribution with the given scale and shape parameters. 🌈
    • float scale — scale parameter
    • float shape — shape parameter
    • int number — number of values to be returned
  • ::math::statistics::random-gumbel location scale number — returns a list of number random values satisfying a Gumbel distribution with the given location and scale parameters. 💅
    • float location — location parameter
    • float scale — scale parameter
    • int number — number of values to be returned
  • ::math::statistics::random-pareto scale shape number — returns a list of number random values satisfying a Pareto distribution with the given scale and shape parameters. 🦋
    • float scale — scale parameter
    • float shape — shape parameter
    • int number — number of values to be returned
  • ::math::statistics::random-cauchy location scale number — returns a list of number random values satisfying a Cauchy distribution with the given location and scale parameters. 🧚
    • float location — location parameter
    • float scale — scale parameter
    • int number — number of values to be returned
  • ::math::statistics::random-laplace location scale number — returns a list of number random values satisfying a Laplace distribution with the given location and shape parameters. again, the Laplace distribution is literally two exponential functions, it's peaked and has heavier tails than the normal distribution. 💖
    • float location — location parameter (mean) 📍
    • float scale — shape parameter 🦋
    • int number — number of values to be returned
  • ::math::statistics::random-kumaraswamy a b number — returns a list of number random values satisfying a Kumaraswamy distribution with the given parameters a and b. related to the Beta distribution but with a tractable cumulative distribution function, which, again, is the entire vibe. 🔥
    • float a — parameter a
    • float b — parameter b
    • int number — number of values to be returned
  • ::math::statistics::random-negative-binomial r p number — returns a list of number random values satisfying a negative binomial distribution. 🍓
    • int r — allowed number of failures (at least 1)
    • float p — probability of success
    • int number — number of values to be returned

🫶 histogram + the fancy integrals + one estimation 🫶

  • ::math::statistics::histogram-uniform xmin xmax limits number — returns the expected histogram for a uniform distribution. like, the histogram you'd predict, not the one you measured. 💜
    • float xmin — minimum value of the distribution
    • float xmax — maximum value of the distribution
    • list limits — upper limits for the buckets in the histogram
    • int number — total number of "observations" in the histogram
  • ::math::statistics::incompleteGamma x p ?tol? — evaluates the incomplete Gamma integral:
                    1       / x               p-1
      P(p,x) =  --------   |   dt exp(-t) * t
                Gamma(p)  / 0
  • float x — value of x (limit of the integral)
  • float p — value of p in the integrand
  • float tol — required tolerance (default: 1.0e-9)
  • ::math::statistics::incompleteBeta a b x ?tol? — evaluates the incomplete Beta integral. ✨
    • float a — first shape parameter
    • float b — second shape parameter
    • float x — value of x (limit of the integral)
    • float tol — required tolerance (default: 1.0e-9)
  • ::math::statistics::estimate-pareto values — estimates the parameters for the Pareto distribution that comes closest to the given values. returns the estimated scale and shape parameters, plus the standard error for the shape parameter. 🎀
    • list values — the values to fit the Pareto distribution to
  • list values — the list of values, assumed to be distributed according to a Pareto distribution

::math::statistics::estimate-exponential values

Estimates the parameter for the exponential distribution that comes closest to the values you hand it. Gives you back an estimate of the one parameter plus the standard error. Two-for-one, bestie 💖

  • list values — the list of values, assumed to be distributed according to an exponential distribution

::math::statistics::estimate-laplace values

Estimates the parameters for the Laplace distribution that comes closest to your values. Returns an estimate of the location parameter and the scale parameter respectively, based on maximum likelihood. Very demure, very maximum likelihood ✨

  • list values — the list of values, assumed to be distributed according to an exponential distribution
okay so yes, that description really does say "exponential distribution" for a Laplace estimator. That's what the page says, so that's what you get. Not me, not my typo, no cap 🙈

::math::statistics::estimante-negative-binomial r values

Estimates the probability of success for the negative binomial distribution that comes closest to the given values. Heads up: you have to give the allowed number of failures too.

  • int r — allowed number of failures, and it has to be at least 1
  • int number — the list of values, assumed to be distributed according to a negative binomial distribution
And yes, the command name is spelled estimante-negative-binomial — that extra e is living its best life and we are not correcting it. Type it exactly like that or it will not run, fr fr 💀

TO DO: more function descriptions to be added

🦋 DATA MANIPULATION 🦋

These data manipulation procedures act on lists, or on lists of lists. Little glow-ups for your data, essentially ✨

::math::statistics::filter varname data expression

Returns a list made up of just the data for which the logical expression is true. Works analogously to the command foreach, if that helps your brain hold it 🫶

  • string varname — name of the variable used in the expression
  • list data — list of data
  • string expression — logical expression using the variable name

::math::statistics::map varname data expression

Returns a list made up of the data that got transformed via the expression. It's giving glow-up montage 💅

  • string varname — name of the variable used in the expression
  • list data — list of data
  • string expression — expression used to transform (map) the data

::math::statistics::samplescount varname list expression

Returns a list of the counts of all the data in the sublists of the list argument for which the expression is true.

  • string varname — name of the variable used in the expression
  • list data — list of sublists, each one containing the data
  • string expression — logical expression to test the data (defaults to "true")

::math::statistics::subdivide

Routine PM — not implemented yet. She's still getting ready, give her a minute 🎀

🌈 PLOT PROCEDURES 🌈

The following simple plotting procedures are available. Yes queen, we're plotting 📈

::math::statistics::plot-scale canvas xmin xmax ymin ymax

Sets the scale for a plot inside the given canvas. Every plot routine expects you to call this one first. There is no automatic scaling provided — you're the main character here, you set the scale yourself 👑

  • widget canvas — canvas widget to use
  • float xmin — minimum x value
  • float xmax — maximum x value
  • float ymin — minimum y value
  • float ymax — maximum y value

::math::statistics::plot-xydata canvas xdata ydata tag

Creates a simple XY plot in the given canvas — the data show up as a little collection of dots. The tag can be used to manipulate the appearance afterwards 🎀

  • widget canvas — canvas widget to use
  • float xdata — series of independent data
  • float ydata — series of dependent data
  • string tag — tag to give to the plotted data (defaults to xyplot)

::math::statistics::plot-xyline canvas xdata ydata tag

Creates a simple XY plot in the given canvas — the data show up as a line running through the data points. The tag can be used to manipulate the appearance.

  • widget canvas — canvas widget to use
  • list xdata — series of independent data
  • list ydata — series of dependent data
  • string tag — tag to give to the plotted data (defaults to xyplot)

::math::statistics::plot-tdata canvas tdata tag

Creates a simple XY plot in the given canvas — the data show up as a collection of dots. The horizontal coordinate equals the index. The tag can be used to manipulate the appearance. This type of presentation is suitable for autocorrelation functions, for instance, or for inspecting time-dependent behaviour. Very "look at how my signal behaves over time" of it ⭐

  • widget canvas — canvas widget to use
  • list tdata — series of dependent data
  • string tag — tag to give to the plotted data (defaults to xyplot)

::math::statistics::plot-tline canvas tdata tag

Creates a simple XY plot in the given canvas — the data show up as a line. See plot-tdata for the explanation, bestie, it's the same energy but connected 🦋

  • widget canvas — canvas widget to use
  • list tdata — series of dependent data
  • string tag — tag to give to the plotted data (defaults to xyplot)

::math::statistics::plot-histogram canvas counts limits tag

Creates a simple histogram in the given canvas 📊

  • widget canvas — canvas widget to use
  • list counts — series of bucket counts
  • list limits — series of upper limits for the buckets
  • string tag — tag to give to the plotted data (defaults to xyplot)

💜 THINGS TO DO 💜

The following procedures are yet to be implemented. Manifesting them, but for now they're just vibes 🧚

  • F-test-stdev
  • interval-mean-stdev
  • histogram-normal
  • histogram-exponential
  • test-histogram
  • test-corr
  • quantiles-*
  • fourier-coeffs
  • fourier-residuals
  • onepar-function-fit
  • onepar-function-residuals
  • plot-linear-model
  • subdivide

🌸 EXAMPLES 🌸

okay bestie, this is the part where it all comes together. here's a whole lil program that shows you how to actually poke at a dataset with this package 💖

# Simple example:
# - Generate data (as a cheap way of getting some)
# - Perform statistical analysis to describe the data
#
package require math::statistics

#
# Two auxiliary procs
#
proc pause {time} {
   set wait 0
   after [expr {$time*1000}] {set ::wait 1}
   vwait wait
}

proc print-histogram {counts limits} {
   foreach count $counts limit $limits {
      if { $limit != {} } {
         puts [format "<%12.4g\t%d" $limit $count]
         set prev_limit $limit
      } else {
         puts [format ">%12.4g\t%d" $prev_limit $count]
      }
   }
}

#
# Our source of arbitrary data
#
proc generateData { data1 data2 } {
   upvar 1 $data1 _data1
   upvar 1 $data2 _data2

   set d1 0.0
   set d2 0.0
   for { set i 0 } { $i < 100 } { incr i } {
      set d1 [expr {10.0-2.0*cos(2.0*3.1415926*$i/24.0)+3.5*rand()}]
      set d2 [expr {0.7*$d2+0.3*$d1+0.7*rand()}]
      lappend _data1 $d1
      lappend _data2 $d2
   }
   return {}
}

#
# The analysis session
#
package require Tk
console show
canvas .plot1
canvas .plot2
pack   .plot1 .plot2 -fill both -side top

generateData data1 data2

puts "Basic statistics:"
set b1 [::math::statistics::basic-stats $data1]
set b2 [::math::statistics::basic-stats $data2]
foreach label {mean min max number stdev var} v1 $b1 v2 $b2 {
   puts "$label\t$v1\t$v2"
}
puts "Plot the data as function of \"time\" and against each other"
::math::statistics::plot-scale .plot1  0 100  0 20
::math::statistics::plot-scale .plot2  0 20   0 20
::math::statistics::plot-tline .plot1 $data1
::math::statistics::plot-tline .plot1 $data2
::math::statistics::plot-xydata .plot2 $data1 $data2

puts "Correlation coefficient:"
puts [::math::statistics::corr $data1 $data2]

pause 2
puts "Plot histograms"
.plot2 delete all
::math::statistics::plot-scale .plot2  0 20 0 100
set limits         [::math::statistics::minmax-histogram-limits 7 16]
set histogram_data [::math::statistics::histogram $limits $data1]
::math::statistics::plot-histogram .plot2 $histogram_data $limits

puts "First series:"
print-histogram $histogram_data $limits

pause 2
set limits         [::math::statistics::minmax-histogram-limits 0 15 10]
set histogram_data [::math::statistics::histogram $limits $data2]
::math::statistics::plot-histogram .plot2 $histogram_data $limits d2
.plot2 itemconfigure d2 -fill red

puts "Second series:"
print-histogram $histogram_data $limits

puts "Autocorrelation function:"
set  autoc [::math::statistics::autocorr $data1]
puts [::math::statistics::map $autoc {[format "%.2f" $x]}]
puts "Cross-correlation function:"
set  crossc [::math::statistics::crosscorr $data1 $data2]
puts [::math::statistics::map $crossc {[format "%.2f" $x]}]

::math::statistics::plot-scale .plot1  0 100 -1  4
::math::statistics::plot-tline .plot1  $autoc "autoc"
::math::statistics::plot-tline .plot1  $crossc "crossc"
.plot1 itemconfigure autoc  -fill green
.plot1 itemconfigure crossc -fill yellow

puts "Quantiles: 0.1, 0.2, 0.5, 0.8, 0.9"
puts "First:  [::math::statistics::quantiles $data1 {0.1 0.2 0.5 0.8 0.9}]"
puts "Second: [::math::statistics::quantiles $data2 {0.1 0.2 0.5 0.8 0.9}]"

so if you actually run this bad boy, these things should be crystal clear to you:

  • There is a strong correlation between two time series, as displayed by the raw data and especially by the correlation functions. 💅
  • Both time series show a significant periodic component 🌊
  • The histograms are not very useful in identifying the nature of the time series - they do not show the periodic nature. (rip histograms, they tried) 🫶

💀 BUGS, IDEAS, FEEDBACK 💀

ngl bestie, this document, and the package it describes, will undoubtedly contain bugs and other problems. fr fr, it's basically guaranteed. Please report such in the category math :: statistics of the Tcllib Trackers at http://core.tcl.tk/tcllib/reportlist. Please also report any ideas for enhancements you may have for either package and/or documentation. we wanna hear it all 👑

When proposing code changes, please provide unified diffs, i.e the output of diff -u. no cap, that's the format we need.

Note further that attachments are strongly preferred over inlined patches. Attachments can be made by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar. left-most button, don't forget it ✨

🔥 KEYWORDS 🔥

data analysis, mathematics, statistics

👑 CATEGORY 👑

Mathematics

and that's a wrap on the whole page, bestie!! this lil slice of statistical excellence is from tcllib version 1.6.1, and it's the math::statistics(3tcl) manual page. go forth and crunch those numbers 💖✨🎀🦋

📦 where this page comes from