math::optimize(3tcl)

📦
version 2.0 · 2 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 math::optimize — the Tcl Math Library glow-up 💖

okay bestie, welcome to math::optimize(3tcl), aka the part of the Tcl Math Library that finds the best possible answer when you're like "idk, somewhere in here??" 🎯✨

✨ NAME ✨

math::optimize — Optimisation routines 🦋

no cap, that's the whole vibe. it optimises. it's giving "make the number as small or as big as it can possibly go."

🌸 SYNOPSIS 🌸

first things first, you gotta have the right Tcl, bestie:

package require Tcl 8.5 9

and then the package itself (version 1.0 is the suggested one, but that's optional, it's in the ?…? so it's a maybe):

package require math::optimize ?1.0?

now the actual commands, in all their iconic glory:

::math::optimize::minimum begin end func maxerr

::math::optimize::maximum begin end func maxerr

::math::optimize::min_bound_1d   func   begin   end  ?-relerror  reltol?
?-abserror abstol? ?-maxiter maxiter? ?-trace traceflag?

::math::optimize::min_unbound_1d  func  begin  end  ?-relerror   reltol?
?-abserror abstol? ?-maxiter maxiter? ?-trace traceflag?

::math::optimize::solveLinearProgram objective constraints

::math::optimize::linearProgramMaximum objective result

::math::optimize::nelderMead  objective  xVector  ?-scale  xScaleVector?
?-ftol epsilon? ?-maxiter count? ??-trace? flag?
fr fr, look at all those ?…? — those mean "optional, bestie, only if you want it." 💅

🎀 DESCRIPTION 🎀

so this package is basically a little collection of optimisation algorithms, and it implements several of them:

  • Minimize or maximize a function over a given interval 🏔️
  • Solve a linear program (maximize a linear function subject to linear constraints) 📐
  • Minimize a function of several variables given an initial guess for the location of the minimum 🔍

and here's the tea: the package is fully implemented in Tcl. no sneaky C, no compiled magic, just vibes and Tcl, bestie 💖

BUT. and this is important, so listen — no particular attention has been paid to the accuracy of the calculations. 😭 instead, the algorithms have been used in a straightforward manner. like, they work, they're chill, they're not out here chasing the last decimal place. periodt.

this document describes the procedures and explains their usage. 📖✨

iykyk: if you need precise numbers for something serious, this is not the moment to be trusting the algorithm's little heart. it's straightforward. it said what it said. 🙈

okay so that's the setup, bestie — more coming, don't go anywhere 🫶

✨ PROCEDURES ✨

ok so this is the part where the package actually does stuff 💅 these are all the public procedures you get to call, and bestie, there are a lot of them. buckle up.

🎀 ::math::optimize::minimum

::math::optimize::minimum begin end func maxerr

Minimizes a continuous function by snooping around the values inside the interval you give it. Here's the vibe: it checks the values at both ends and in the centre of the interval, then builds a brand-new interval that's half as long and still contains the minimum. rinse and repeat, main character energy.

⚠️ no cap: there is no guarantee that the global minimum is what you'll get. it finds a minimum, not necessarily the one. 💀

It hands back the "x" value where the function is smallest.

🚫 this procedure has been deprecated — use min_bound_1d instead, fr fr.
  • begin — where the interval starts, bestie.
  • end — where the interval ends.
  • func — the name of the function you wanna minimize (it's a procedure that takes one argument).
  • maxerr — maximum relative error, and it defaults to 1.0e-4.

🎀 ::math::optimize::maximum

::math::optimize::maximum begin end func maxerr

Same situation but flipped — it maximizes a continuous function by examining the values in the interval. It peeks at both ends and the centre, then constructs a new interval of half the length that includes the maximum. slay.

⚠️ again, no guarantee the global maximum is found. it's giving "best effort," not "certified." 😭

Returns the "x" value where the function is maximal.

🚫 also deprecated — use max_bound_1d instead.
  • begin — start of the interval.
  • end — end of the interval.
  • func — name of the function to be maximized (a procedure taking one argument).
  • maxerr — maximum relative error (defaults to 1.0e-4).

🦋 ::math::optimize::min_bound_1d

::math::optimize::min_bound_1d func begin end ?-relerror reltol?
?-abserror abstol? ?-maxiter maxiter? ?-trace traceflag?

Minimizes a function of one variable inside the interval you give it. The method? Brent's method of parabolic interpolation, protected by golden-section subdivisions whenever the interpolation is lowkey not converging. iconic behavior. ✨

⚠️ still no guarantee a global minimum is found, bestie. it's very good, not psychic.

The function you're evaluating, func, has to be a single Tcl command; it gets evaluated with an abscissa appended as the last argument.

x1 and x2 are the two bounds of the interval where the minimum lives. and guess what — they do NOT need to be in increasing order. so chill. 🫶

reltol, if you specify it, is the desired upper bound on the relative error of the result, default 1.0e-7. key thing: the value you give should never be smaller than the square root of the machine's floating point precision, or else convergence is not guaranteed. periodt.

abstol, if specified, is the desired upper bound on the absolute error of the result, default 1.0e-10. 🚨 caution must be used with small values of abstol to avoid overflow/underflow conditions; if the minimum is expected to lie about a small but non-zero abscissa, you should consider either shifting the function or changing its length scale.

maxiter constrains the number of function evaluations performed; default is 100. If the command evaluates the function more than maxiter times, it returns an error to the caller. 🙈

traceFlag is a Boolean value. If true, it makes the command print a message on the standard output giving the abscissa and ordinate at each function evaluation, together with an indication of what type of interpolation was chosen. Default is 0 (no trace).

🧚 ::math::optimize::min_unbound_1d

::math::optimize::min_unbound_1d func begin end ?-relerror reltol?
?-abserror abstol? ?-maxiter maxiter? ?-trace traceflag?

Same energy as above but it minimizes a function of one variable over the entire real number line 🌈 no interval holding it back.

The method is a whole journey: parabolic extrapolation combined with golden-section dilatation to hunt down a region where a minimum actually exists, followed by Brent's method of parabolic interpolation, protected by golden-section subdivisions if the interpolation is not converging. 💅

⚠️ and yes, still no guarantee that a global minimum is found. we stay honest here.

func must be a single Tcl command; it will be evaluated with an abscissa appended as the last argument.

x1 and x2 are two initial guesses at where the minimum may lie. x1 is the starting point for the minimization, and the difference between x2 and x1 is used as a hint at the characteristic length scale of the problem.

reltol, if specified, is the desired upper bound on the relative error of the result; default 1.0e-7. again: the given value should never be smaller than the square root of the machine's floating point precision, or else convergence is not guaranteed.

abstol, if specified, is the desired upper bound on the absolute error of the result; default 1.0e-10. 🚨 same caution — small values of abstol risk overflow/underflow; if the minimum is expected to lie about a small but non-zero abscissa, consider either shifting the function or changing its length scale.

maxiter constrains the number of function evaluations; default is 100. Evaluate more than maxiter times and it returns an error to the caller. 😭

traceFlag is a Boolean. If true, it prints a message on the standard output giving the abscissa and ordinate at each function evaluation, plus an indication of what type of interpolation was chosen. Default is 0 (no trace).

👑 ::math::optimize::solveLinearProgram

::math::optimize::solveLinearProgram objective constraints

Solves a linear program in standard form using a straightforward implementation of the Simplex algorithm. and just so we're all on the same page for the explanation below: the linear program has N constraints and M variables. 💖

The procedure returns a list of M values — the values for which the objective function is maximal — or a single keyword if the linear program is not feasible or unbounded (either "unfeasible" or "unbounded"). ⭐

  • objective — the M coefficients of the objective function.
  • constraints — matrix of coefficients plus maximum values that implement the linear constraints. It's expected to be a list of N lists of M+1 numbers each, M coefficients and the maximum value.

🍓 ::math::optimize::linearProgramMaximum

::math::optimize::linearProgramMaximum objective result

A lil convenience function 🫶 — it just returns the maximum for the solution found by the solveLinearProgram procedure.

  • objective — the M coefficients of the objective function.
  • result — the result as returned by solveLinearProgram.

💜 ::math::optimize::nelderMead

::math::optimize::nelderMead objective xVector ?-scale xScaleVector?
?-ftol epsilon? ?-maxiter count? ??-trace? flag?

Minimizes, in unconstrained fashion, a function of several variables over all of space. 🌌 the function to evaluate, objective, must be a single Tcl command. To it will be appended as many elements as appear in the initial guess at the location of the minimum, which you pass in as a Tcl list, xVector.

xScaleVector is an initial guess at the problem scale; the first function evaluations will be made by varying the co-ordinates in xVector by the amounts in xScaleVector. If xScaleVector is not supplied, the co-ordinates will be varied by a factor of 1.0001 (if the co-ordinate is non-zero) or by a constant 0.0001 (if the co-ordinate is zero). 💅

epsilon is the desired relative error in the value of the function evaluated at the minimum. The default is 1.0e-7, which usually gives three significant digits of accuracy in the values of the x's.

pp count is a limit on the number of trips through the main loop of the optimizer. The number of function evaluations may be several times this number. If the optimizer fails to find a minimum to within ftol in maxiter iterations, it returns its current best guess and an error status. Default is to allow 500 iterations. 😮‍💨

flag is a flag that, if true, causes a line to be written to the standard output for each evaluation of the objective function, giving the arguments presented to the function and the value returned. Default is false.

The nelderMead procedure returns a list of alternating keywords and values suitable for use with array set. The meaning of the keywords is:

  • x — the approximate location of the minimum.
  • y — the value of the function at x.
  • yvec — a vector of the best N+1 function values achieved, where N is the dimension of x.
  • vertices — a list of vectors giving the function arguments corresponding to the values in yvec.
  • nIter — the number of iterations required to achieve convergence or fail.
  • status — 'ok' if the operation succeeded, or 'too-many-iterations' if the maximum iteration count was exceeded.

nelderMead minimizes the given function using the downhill simplex method of Nelder and Mead. This method is quite slow — much faster methods for minimization are known — but it has the advantage of being extremely robust in the face of problems where the minimum lies in a valley of complex topology. 🏔️ iconic resilience honestly.

nelderMead can occasionally find itself "stuck" at a point where it can make no further progress; it is recommended that the caller run it at least a second time, passing as the initial guess the result found by the previous call. The second run is usually very fast. ⚡

nelderMead can be used in some cases for constrained optimization. To do this, add a large value to the objective function if the parameters are outside the feasible region. To work effectively in this mode, nelderMead requires that the initial guess be feasible and usually requires that the feasible region be convex.

🦋 NOTES (read this bestie, it's important) 🦋

Okay so several of the procedures above take the names of procedures as their arguments. Plot twist: to avoid any drama with the visibility of those procedures, the fully-qualified name of them gets figured out inside the optimize routines. For you, that means exactly one consequence, and it's giving very simple: the named procedure has to be visible in the calling procedure. ✨

Like, for instance:

namespace eval ::mySpace {
   namespace export calcfunc
   proc calcfunc { x } { return $x }
}
#
# Use a fully-qualified name
#
namespace eval ::myCalc {
   puts [min_bound_1d ::myCalc::calcfunc $begin $end]
}
#
# Import the name
#
namespace eval ::myCalc {
   namespace import ::mySpace::calcfunc
   puts [min_bound_1d calcfunc $begin $end]
}

And bestie, listen — the simple procedures minimum and maximum have been deprecated 😭. The alternatives are so much more flexible, more robust, and they need fewer function evaluations. It's giving glow-up, honestly. 💅

🌸 EXAMPLES 🌸

Let's do a few simple ones, okay? Grab a snack. 🍓

Determine the maximum of f(x) = x^3 exp(-3x), on the interval (0,10):

proc efunc { x } { expr {$x*$x*$x * exp(-3.0*$x)} }
puts "Maximum at: [::math::optimize::max_bound_1d efunc 0.0 10.0]"

The maximum allowed error decides how many steps get taken (and with each step in the iteration, the interval is reduced with a factor of 1/2). So a maximum error of 0.0001 is achieved in approximately 14 steps. Math ate that fr fr. 🔥

An example of a linear program is:

Optimise the expression 3x+2y, where:

  • x >= 0 and y >= 0 (these are the implicit constraints, part of the definition of linear programs — they come with the territory, no cap)
  • x + y <= 1 (constraints specific to the problem)
  • 2x + 5y <= 10

This problem can be solved as follows:

set solution [::math::optimize::solveLinearProgram  { 3.0   2.0 }  { { 1.0   1.0   1.0 }
     { 2.0   5.0  10.0 } } ]

Note, that a constraint like:

x + y >= 1

can be turned into standard form using:

-x  -y <= -1

The theory of linear programming is the subject of many a text book, and the Simplex algorithm that is implemented here is the best-known method to solve this type of problems — but it is not the only one. Main character algorithm behavior, honestly. 👑

💀 BUGS, IDEAS, FEEDBACK 💀

This document, and the package it describes, will undoubtedly contain bugs and other problems. Not me manifesting that but okay — it's true of everything. Please report such in the category math :: optimize of the Tcllib Trackers at http://core.tcl.tk/tcllib/reportlist. 🫶

Please also report any ideas for enhancements you may have for either the package and/or the documentation. We want the ideas, bestie, we're obsessed with them. ✨

When proposing code changes, please provide unified diffs, i.e the output of diff -u.

Note further that attachments are strongly preferred over inlined patches. Attachments can be made by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar. (iykyk 🧚)

⭐ KEYWORDS ⭐

linear program, math, maximum, minimum, optimization

🌈 CATEGORY 🌈

Mathematics

Copyright (c) 2004 Arjen Markus <arjenmarkus@users.sourceforge.net>
Copyright (c) 2004,2005 Kevn B. Kenny <kennykb@users.sourceforge.net>

and that's the whole page, bestie!! 🎀 This is math::optimize(3tcl) from tcllib 1.0 — go forth and optimise literally everything, you absolute icon. 🦋💖✨

📦 where this page comes from