funcalc(1)

✨ NAME ✨

funcalc — the Funtools calculator (for binary tables) 💖

---

🎀 SYNOPSIS 🎀

funcalc [-n] [-a argstr] [-e expr] [-f file] [-l link] [-p prog] <iname>
[oname [columns]]

---

💅 OPTIONS 💅

• -a argstr — user arguments you wanna pass to the compiled program, bestie.
• -e expr — the funcalc expression itself.
• -f file — a file that contains your funcalc expression.
• -l libs — libs to add to the link command.
• -n — output the generated code instead of compiling and executing it (peek behind the curtain ✨).
• -p prog — generate a named program, no execution. just vibes, no running.
• -u — die if any variable is undeclared (i.e. don't auto-declare it). strict mode energy 💀.

---

🦋 DESCRIPTION 🦋

funcalc is a calculator program that lets you build arbitrary expressions, compile them, and execute them on columns in a Funtools table (that's a FITS binary table or a raw event file, iykyk). The way it works is so cute: it takes your user-supplied expression(s) and slots them into a template C program, then compiles and runs that program. funcalc expressions are literally C statements, though there are some important simplifications (like automatic declaration of variables) that we support for you. 🫶

funcalc expressions can be specified in three different ways:

1. On the command line using the -e [expression] switch.
2. In a file using the -f [file] switch.
3. From stdin (if neither -e nor -f is specified).

And of course a file containing funcalc expressions can be read from stdin. no cap. 🍓

Every single invocation of funcalc needs an input Funtools table file, and that's specified as the first command line argument. The output Funtools table file is the second optional argument — it's only needed if you're actually creating an output FITS file (so like, in cases where the funcalc expression only prints values, you don't need an output file at all). If input and output file are both specified, a third optional argument can specify the list of columns to activate (using FunColumnActivate()). Note that funcalc figures out whether or not to generate code for writing an output file based on the presence or absence of an output file argument. ✨

A funcalc expression executes on each row of a table and consists of one or more C statements that operate on the columns of that row (possibly using temporary variables). Inside an expression, you reference a column of the current row using the C struct syntax cur-[colname]>, e.g. cur->x, cur->pha, etc. Local scalar variables can be defined using C declarations at the very beginning of the expression, or else they can be defined automatically by funcalc (as type double). So for example, a swap of columns x and y in a table can be done with either of these equivalent funcalc expressions:

double temp;
temp = cur->x;
cur->x = cur->y;
cur->y = temp;

or:

temp = cur->x;
cur->x = cur->y;
cur->y = temp;

When this expression is executed using a command such as:

funcalc -f swap.expr itest.ev otest.ev

the resulting file will have values of the x and y columns swapped. iconic. 💅

By default, the data type of the variable for a column is the same as the data type of the column as stored in the file. You can change this by appending :[dtype] to the first reference to that column. In the example above, to force x and y to be output as doubles, specify the type 'D' explicitly:

temp = cur->x:D;
cur->x = cur->y:D;
cur->y = temp;

Data type specifiers follow the standard FITS table syntax for defining columns using TFORM:

• A: ASCII characters
• B: unsigned 8-bit char
• I: signed 16-bit int
• U: unsigned 16-bit int (not standard FITS)
• J: signed 32-bit int
• V: unsigned 32-bit int (not standard FITS)
• E: 32-bit float
• D: 64-bit float
• X: bits (treated as an array of chars)

Note that only the first reference to a column should contain the explicit data type specifier. 👑

Of course, it is so important to handle the data type of the columns correctly. One of the most frequent causes of error in funcalc programming is the implicit use of the wrong data type for a column in an expression. For example, this calculation:

dx = (cur->x - cur->y)/(cur->x + cur->y);

usually needs to be performed using floating point arithmetic. In cases where the x and y columns are integers, you can do this by reading the columns as doubles using an explicit type specification:

dx = (cur->x:D - cur->y:D)/(cur->x + cur->y);

Alternatively, you can do it using C type-casting in the expression:

dx = ((double)cur->x - (double)cur->y)/((double)cur->x + (double)cur->y);

In addition to accessing columns in the current row, you can also reference the previous row using prev-[colname]>, and the next row using next-[colname]>. Heads up though: if prev-[colname]> is specified in the funcalc expression, the very first row is not processed. And if next-[colname]> is specified in the funcalc expression, the very last row is not processed. This way, prev and next are guaranteed to always point to valid rows. 🌈 For example, to print out the values of the current x column and the previous y column, use the C fprintf function in a funcalc expression:

fprintf(stdout, "%d %d\n", cur->x, prev->y);

New columns can be specified using the same cur-[colname]> syntax by appending the column type (and optional tlmin/tlmax/binsiz specifiers), separated by colons. For example, cur->avg:D will define a new column of type double. Type specifiers are the same as the ones used above to specify new data types for existing columns.

For example, to create and output a new column that is the average value of the x and y columns, a new "avg" column can be defined:

cur->avg:D = (cur->x + cur->y)/2.0

Note that the final ; is not required for single-line expressions. 🎀

As with FITS TFORM data type specification, the column data type specifier can be preceded by a numeric count to define an array, e.g. "10I" means a vector of 10 short ints, "2E" means two single precision floats, etc. A new column only needs to be defined once in a funcalc expression, after which it can be used without re-specifying the type. This includes reference to elements of a column array:

cur->avg[0]:2D = (cur->x + cur->y)/2.0;
cur->avg[1] = (cur->x - cur->y)/2.0;

The 'X' (bits) data type is treated as a char array of dimension (numeric_count/8), i.e. 16X is processed as a 2-byte char array. Each 8-bit array element is accessed separately:

cur->stat[0]:16X  = 1;
cur->stat[1]      = 2;

Here, a 16-bit column is created with the MSB is set to 1 and the LSB set to 2. 💫

By default, all processed rows are written to the specified output file. If you wanna skip writing certain rows, just execute the C "continue" statement at the end of the funcalc expression, since the writing of the row is performed immediately after the expression is executed. For example, to skip writing rows whose average is the same as the current x value:

cur->avg[0]:2D = (cur->x + cur->y)/2.0;
cur->avg[1] = (cur->x - cur->y)/2.0;
if( cur->avg[0] == cur->x )
  continue;

If no output file argument is specified on the funcalc command line, no output file is opened and no rows are written. This is useful in expressions that simply print output results instead of generating a new file:

fpv = (cur->av3:D-cur->av1:D)/(cur->av1+cur->av2:D+cur->av3);
fbv =  cur->av2/(cur->av1+cur->av2+cur->av3);
fpu = ((double)cur->au3-cur->au1)/((double)cur->au1+cur->au2+cur->au3);
fbu =  cur->au2/(double)(cur->au1+cur->au2+cur->au3);
fprintf(stdout, "%f\t%f\t%f\t%f\n", fpv, fbv, fpu, fbu);

In the above example, we use both explicit type specification (for "av" columns) and type casting (for "au" columns) to make sure all operations are performed in double precision. 🔥

When an output file is specified, the selected input table is processed and output rows are copied to the output file. Note that the output file can be specified as "stdout" in order to write the output rows to the standard output. If the output file argument is passed, an optional third argument also can be passed to specify which columns to process.

In a FITS binary table, it sometimes is desirable to copy all of the other FITS extensions to the output file as well. You can do this by appending a '+' sign to the name of the extension in the input file name. See funtable for a related example. 🌸

funcalc works by integrating the user-specified expression into a template C program called tabcalc.c. The completed program then is compiled and executed. Variable declarations that begin the funcalc expression are placed in the local declaration section of the template main program. All other lines are placed in the template main program's inner processing loop. Other details of program generation are handled automatically. For example, column specifiers are analyzed to build a C struct for processing rows, which is passed to FunColumnSelect() and used in FunTableRowGet(). If an unknown variable is used in the expression, resulting in a compilation error, the program build is retried after defining the unknown variable to be of type double. (it just knows 💅)

Normally, funcalc expression code is added to the funcalc row processing loop. It is possible to add code to other parts of the program by placing this code inside special directives of the form:

[directive name]
  ... code goes here ...
end

The directives are:

• global — add code and declarations in global space, before the main routine.
• local — add declarations (and code) just after the local declarations in main
• before — add code just before entering the main row processing loop
• after — add code just after exiting the main row processing loop

Thus, the following funcalc expression will declare global variables and make subroutine calls just before and just after the main processing loop:

global
  double v1, v2;
  double init(void);
  double finish(double v);
end
before
  v1  = init();
end
... process rows, with calculations using v1 ...
after
  v2 = finish(v1);
  if( v2 < 0.0 ){
    fprintf(stderr, "processing failed %g -> %g\n", v1, v2);
    exit(1);
  }
end

Routines such as init() and finish() above are passed to the generated program for linking using the -l [link directives ...] switch. The string specified by this switch will be added to the link line used to build the program (before the funtools library). For example, assuming that init() and finish() are in the library libmysubs.a in the /opt/special/lib directory, use:

funcalc  -l "-L/opt/special/lib -lmysubs" ...

okay bestie, so — you can absolutely yeet your own arguments into a compiled funcalc program, and you do it with a string argument to the -a switch. That one string holds all of your user arguments. So if you wanna pass the integers 1 and 2, it's giving:

funcalc -a "1 2" ...

Those arguments get stashed in an internal array, and you reach them as strings through the ARGV(n) macro. Like, say you wrote this lil expression:

local
  int pmin, pmax;
end

before
  pmin=atoi(ARGV(0));
  pmax=atoi(ARGV(1));
end

if( (cur->pha >= pmin) && (cur->pha <= pmax) )
  fprintf(stderr, "%d %d %d\n", cur->x, cur->y, cur->pha);

✨ This expression prints out the x, y, and pha values for every row where pha lands between the two user-input values:

funcalc -a '1 12' -f foo snr.ev'[cir 512 512 .1]'
512 512 6
512 512 8
512 512 5
512 512 5
512 512 8

funcalc -a '5 6' -f foo snr.ev'[cir 512 512 .1]'
512 512 6
512 512 5
512 512 5

🫶 Heads up though, bestie: it is 100% the user's responsibility to make sure the correct number of arguments gets passed. The ARGV(n) macro returns a NULL if the argument you asked for is outside the limits of the actual number of args — which usually results in a SEGV if processed blindly. Not cute. Not the vibe. 💀

To check the argument count, use the ARGC macro:

local
  long int seed=1;
  double limit=0.8;
end

before
  if( ARGC >= 1 ) seed = atol(ARGV(0));
  if( ARGC >= 2 ) limit = atof(ARGV(1));
  srand48(seed);
end

if ( drand48() > limit ) continue;

---

The macro WRITE_ROW expands to the FunTableRowPut() call that writes the current row. And the way it can write the row more than once? Obsessed. Plus, the macro NROW expands to the row number currently being processed. Both of them in action:

if( cur->pha:I == cur->pi:I ) continue;
a = cur->pha;
cur->pha = cur->pi;
cur->pi = a;
cur->AVG:E  = (cur->pha+cur->pi)/2.0;
cur->NR:I = NROW;
if( NROW < 10 ) WRITE_ROW;

If the -p [prog] switch is specified, the expression is not executed. Instead, the generated executable is saved with the specified program name for later use. 📦

If the -n switch is specified, the expression is also not executed — instead the generated code gets written to stdout. This is especially useful if you wanna generate a skeleton file and add your own code, or if you need to check compilation errors. And ngl, here's a lil gift: the comment at the start of the output gives you the compiler command needed to build the program on that platform. 🎀 (Fair warning, that command can change from platform to platform because of the use of different libraries, compiler switches, etc.)

As mentioned previously, funcalc will declare a scalar variable automatically (as a double) if that variable has been used but not declared. This facility is implemented using a sed script named funcalc.sed, which processes the compiler output to sense an undeclared variable error. This script has been seeded with the appropriate error information for gcc, and for cc on Solaris, DecAlpha, and SGI platforms. So if you find that automatic declaration of scalars is not working on your platform, go check this sed script — it might be necessary to add to or edit some of the error messages it senses. 🔍

---

Okay so, in order to keep the lexical analysis of funcalc expressions (reasonably) simple, they chose to accept some limitations on how accurately C comments, spaces, and new-lines are placed in the generated program. In particular, comments associated with local variables declared at the beginning of an expression (i.e., not in a local...end block) will usually end up in the inner loop, not with the local declarations:

/* this comment will end up in the wrong place (i.e, inner loop) */
double a; /* also in wrong place */
/* this will be in the the right place (inner loop) */
if( cur->x:D == cur->y:D ) continue; /* also in right place */
a = cur->x;
cur->x = cur->y;
cur->y = a;
cur->avg:E  = (cur->x+cur->y)/2.0;

Similarly, spaces and new-lines sometimes are omitted or added in a seemingly arbitrary manner. 😭 But fr fr, none of these stylistic blemishes affect the correctness of the generated code, so we move.

---

Because funcalc must analyze the user expression using the data file(s) passed on the command line, the input file(s) must be opened and read twice: once during program generation and once during execution. As a result, it is not possible to use stdin for the input file — funcalc cannot be used as a filter. 🚫 They will consider removing this restriction at a later time, so like, stay tuned bestie.

Along with C comments, funcalc expressions can have one-line internal comments that are not passed on to the generated C program. These internal comments start with the # character and continue up to the new-line:

double a; # this is not passed to the generated C file
# nor is this
a = cur->x;
cur->x = cur->y;
cur->y = a;
/* this comment is passed to the C file */
cur->avg:E  = (cur->x+cur->y)/2.0;

---

As previously mentioned, input columns normally are identified by their being used within the inner event loop. But there are rare cases where you might wanna read a column and process it outside the main loop. For example, qsort might use a column in its sort comparison routine that is not processed inside the inner loop (and therefore not implicitly specified as a column to be read). To ensure that such a column is read by the event loop, use the explicit keyword. ✨ The arguments to this keyword specify columns that should be read into the input record structure even though they are not mentioned in the inner loop. For example:

explicit pi pha

will ensure that the pi and pha columns are read for each row, even if they are not processed in the inner event loop. The explicit statement can be placed anywhere. 💅

Finally, note that funcalc currently works on expressions involving FITS binary tables and raw event files. They will consider adding support for image expressions at a later point, if there is demand for such support from the community. 👑

---

👀 SEE ALSO 👀

See funtools(7) for a list of Funtools help pages. 🦋

---

and that's the whole page, bestie!! 🎀 this was <b>version 1.4.5</b>, dated <b>April 14, 2011</b> — iconic behavior from the funcalc era, fr. go forth and pass your arguments responsibly 💖✨