mlpack_nca(1)

📦
version 4.8.0 · 5 versions
, in ubuntu stonking

💖 mlpack_nca — the manual page, but make it make sense 💖

okay bestie, gather round, because this is mlpack_nca(1) from the mlpack User Commands lineup and it's genuinely one of the cooler ones. Neighborhood Components Analysis. NCA. Say it with me. It's giving ✨ dimensionality reduction ✨ and distance learning at the same time, which is lowkey iconic of it.

✨ NAME ✨

mlpack_nca — neighborhood components analysis, aka nca, aka your new fave 🫶

🌸 SYNOPSIS 🌸

mlpack_nca -i unknown [-A double] [-b int] [-l unknown] [-L bool] [-n int] [-T int] [-M double] [-m double] [-N bool] [-B int] [-O string] [-s int] [-a double] [-t double] [-V bool] [-w double] [-o unknown] [-h -v]

🦋 DESCRIPTION 🦋

So this program implements Neighborhood Components Analysis, which is both a linear dimensionality reduction technique AND a distance learning technique. Two jobs, one binary, main character energy fr fr.

The method wants to make k-nearest-neighbor classification on a dataset better by scaling the dimensions. And here's the cute part — it's nonparametric, so it doesn't even need you to hand it a value of k. It works by using stochastic ("soft") neighbor assignments and then doing optimization techniques over the gradient of the accuracy of those neighbor assignments. Fancy. Obsessed. 💅

Heads up bestie: to work, this algorithm needs labeled data. Periodt.

You can hand it the labels two ways:

  • as the last row of the input dataset (that's the one you specify with --input_file (-i)), or
  • alternatively as a separate matrix (specified with --labels_file (-l))

This implementation of NCA uses stochastic gradient descent, mini-batch stochastic gradient descent, or the L_BFGS optimizer. Important caveat though — these optimizers do not guarantee global convergence for a nonconvex objective function (and NCA's objective function is nonconvex), so your final results could literally depend on the random seed or other optimizer parameters. Not a bug, just the vibe. 😭

🌈 the SGD saga 🌈

Stochastic gradient descent, which you get by setting the parameter --optimizer (-O) to the value 'sgd', depends primarily on three parameters:

  • the step size — --step_size (-a)
  • the batch size — --batch_size (-b)
  • the maximum number of iterations — --max_iterations (-n)

In addition, you can use a normalized starting point by specifying the --normalize (-N) parameter. And ngl, that one is necessary if you're getting a bunch of warnings of the form Denominator of p_i is 0!

Tuning the step size can be a tedious affair (real). In general:

  • the step size is too large if the objective is not mostly uniformly decreasing, or if zero-valued denominator warnings are being issued
  • the step size is too small if the objective is changing very slowly

Setting the termination condition is easy peasy once you find a good step size parameter. Either:

  1. increase the maximum iterations to a large number and let SGD find a minimum, or
  2. set the maximum iterations to 0 (which allows infinite iterations) and set the tolerance (specified by --tolerance (-t)) to define the maximum allowed difference between objectives for SGD to terminate.
⚠️ Be careful — setting the tolerance instead of the maximum iterations can take a very long time and may actually never converge due to the properties of the SGD optimizer. That's not me being dramatic, that's the manual.

Also, note this: a single iteration of SGD refers to a single point, so to take a single pass over the dataset, set the value of the --max_iterations (-n) parameter equal to the number of points in the dataset. iykyk. 🎀

💜 the L-BFGS lore 💜

The L-BFGS optimizer — which you get by setting the parameter --optimizer (-O) to the value 'lbfgs' — uses a back-tracking line search algorithm to minimize a function.

The following parameters are used by L-BFGS:

  • --num_basis (-B) — specifies the number of memory points used by L-BFGS
  • --max_iterations (-n)
  • --armijo_constant (-A)
  • --wolfe (-w)
  • --tolerance (-t) — the optimization is terminated when the gradient norm is below this value
  • --max_line_search_trials (-T)
  • --min_step (-m)
  • --max_step (-M) — and note that --min_step (-m) and --max_step (-M) both refer to the line search routine

For more details on the L-BFGS optimizer, consult either the mlpack L-BFGS documentation (in lbfgs.hpp) or the vast set of published literature on L-BFGS. Go read, queen. 📚

By default, the SGD optimizer is used.

👑 REQUIRED INPUT OPTIONS 👑

  • --input_file (-i) [unknown] — the input dataset to run NCA on. You literally cannot skip this one, bestie.

✨ OPTIONAL INPUT OPTIONS ✨

  • --armijo_constant (-A) [double] — Armijo constant for L-BFGS. Default value 0.0001.
  • --batch_size (-b) [int] — Batch size for mini-batch SGD. Default value 50.
  • --help (-h) [bool] — Default help info.
  • --info [string] — Print help on a specific option. Default value ''.
  • --labels_file (-l) [unknown] — Labels for input dataset.
  • --linear_scan (-L) [bool] — Don't shuffle the order in which data points are visited for SGD or mini-batch SGD. (Rude of you to skip the shuffle, but okay 💀)
  • --max_iterations (-n) [int] — Maximum number of iterations for SGD or L-BFGS (0 indicates no limit). Default value 500000.
  • --max_line_search_trials (-T) [int] — Maximum number of line search trials for L-BFGS. Default value 50.
  • --max_step (-M) [double] — Maximum step of line search for L-BFGS. Default value 1e+20.
  • --min_step (-m) [double] — Minimum step of line search for L-BFGS. Default value 1e-20.
  • --normalize (-N) [bool] — Use a normalized starting point for optimization. This is useful for when points are far apart, or when SGD is returning NaN. 🚨
  • --num_basis (-B) [int] — Number of memory points to be stored for L-BFGS. Default value 5.
  • --optimizer (-O) [string] — Optimizer to use; 'sgd' or 'lbfgs'. Default value 'sgd'.
  • --seed (-s) [int] — Random seed. If 0, 'std::time(NULL)' is used. Default value 0.
  • --step_size (-a) [double] — Step size for stochastic gradient descent (alpha). Default value 0.01.
  • --tolerance (-t) [double] — Maximum tolerance for termination of SGD or L-BFGS. Default value 1e-07.
  • --verbose (-v) [bool] — Display informational messages and the full list of parameters and timers at the end of execution. Basically the "tell me everything" button. 🗣️
  • --version (-V) [bool] — Display the version of mlpack.
  • --wolfe (-w) [double] — Wolfe condition parameter for L-BFGS. Default value 0.9.

🎀 OPTIONAL OUTPUT OPTIONS 🎀

  • --output_file (-o) [unknown] — Output matrix for learned distance matrix. This is the one you actually want, the whole point, the moment. ⭐

🌟 ADDITIONAL INFORMATION 🌟

For further information, including relevant papers, citations, and theory, consult the documentation found at http://www.mlpack.org or included with your distribution of mlpack. Go forth and read, bestie, you've got this. 💖

and that's the whole page, no cap. this was mlpack-4.8.0, dated 22 June 2026, served fresh from the mlpack_nca(1) manual. slay, bye 👑✨

📦 where this page comes from