funcolumnselect(3)
🦋 funcolumnselect(3) — SAORD Documentation 🦋
okay bestie, sit down, hydrate, and let's talk about picking your columns 💅 this is the Funtools page for FunColumnSelect, and it's a long one so it comes in 2 parts — this is part 1, so don't you dare scroll away 🌸
✨ NAME ✨
FunColumnSelect — select Funtools columns 👑
🎀 SYNOPSIS 🎀
#include <funtools.h>
int FunColumnSelect(Fun fun, int size, char *plist,
char *name1, char *type1, char *mode1, int offset1,
char *name2, char *type2, char *mode2, int offset2,
...,
NULL)
int FunColumnSelectArr(Fun fun, int size, char *plist,
char **names, char **types, char **modes,
int *offsets, int nargs);💜 DESCRIPTION 💜
okay so the FunColumnSelect() routine is literally the one that selects the columns out of a Funtools binary table extension or a raw event file so you can actually do stuff with them. it lets you say exactly how columns in a file get read into a user record structure — or written from a user record structure out to an output FITS file. both directions, no cap 🫶
The first argument is the Fun handle that's associated with this set of columns. The second argument is the size of the user record structure that the columns get read into — and yes, you basically always just use the sizeof() macro for that, it's giving convenience ✨ The third argument is where you drop keyword directives for the selection, and that gets explained in more detail further down.
After those first three required arguments comes a variable length list of column specifications. every single column spec is made of four arguments:
- name — the name of the column 🏷️
- type — the data type of the column as it will be stored in the user record struct, not the data type of the input file, okay? important distinction, iykyk. these are the basic data types it recognizes:
A— ASCII charactersB— unsigned 8-bit charI— signed 16-bit intU— unsigned 16-bit int (not standard FITS 👀)J— signed 32-bit intV— unsigned 32-bit int (not standard FITS either 👀)E— 32-bit floatD— 64-bit float
the syntax is basically the same vibe as whatever defines the TFORM parameter in FITS binary tables. meaning a numeric repeat value can come before the type character, so "10I" means a vector of 10 short ints, "E" means a single precision float, etc. and note that the column value from the input file gets converted to whatever data type you specified as the data is read by FunTableRowGet() 🔥
- 🌈 [ a short digression regarding bit-fields ] — special attention is required when reading or writing the FITS bit-field type (
"X"). bit-fields almost always have a numeric repeat character preceding the'X'spec. usually that value is a multiple of 8 so the bit-fields fit into an integral number of bytes. for all cases, the byte size of the bit-field B is(N+7)/8, where N is the numeric repeat character.
the easiest way to declare a bit-field in the user struct is as an array of type char of size B as defined above. in that case, bytes are simply moved from the file straight into the user space, periodt. if instead you use a short or int scalar or array, then the algorithm for reading the bit-field into user space depends on the size of the data type used plus the value of the repeat character. so: if the user data size equals the byte size of the bit-field, the data is just moved (possibly with endian-based byte-swapping) from one to the other. if, on the other hand, the data storage is larger than the bit-field size, then a data type cast conversion happens to move parts of the bit-field into elements of the array. examples make this so much clearer, so:
- if the file contains a
16Xbit-field and user space specifies a2Bchar array[2], then the bit-field is moved directly into the char array ✨ - if the file contains a
16Xbit-field and user space specifies a1Iscalar short int, then the bit-field is moved directly into the short int ✨ - if the file contains a
16Xbit-field and user space specifies a1Jscalar int, then the bit-field is type-cast to unsigned int before being moved (use of unsigned avoids possible sign extension) ⚠️ - if the file contains a
16Xbit-field and user space specifies a2Jint array[2], then the bit-field is handled as 2 chars, each of which are type-cast to unsigned int before being moved (again, use of unsigned avoids possible sign extension) ⚠️ - if the file contains a
16Xbit-field and user space specifies a1Bchar, then the bit-field is treated as a char, i.e., truncation will occur 💀 - if the file contains a
16Xbit-field and user space specifies a4Jint array[4], then the results are undetermined 🙈
and for all user data types larger than char, the bit-field is byte-swapped as necessary to convert to native format, so that bits in the resulting data in user space can be tested, masked, etc. the same way regardless of platform. slay, portability 💅
so beyond setting data type and size, the type specification also lets you set a few ancillary parameters, using the full syntax for type:
[@][n]<type>[[['B']poff]][:[tlmin[:tlmax[:binsiz]]]]
the special character "@" can be prepended to this spec to indicate that the data element is a pointer in the user record, rather than an array stored inside the record. sneaky but iconic.
the [n] value is an integer specifying the number of elements in this column (default is 1). TLMIN, TLMAX, and BINSIZ values can also be specified for this column after the type, separated by colons. if only one such number is specified, it's assumed to be TLMAX, and TLMIN and BINSIZ are set to 1.
the [poff] value can be used to specify the offset into an array. by default this offset value is set to zero and the data specified starts at the beginning of the array. the offset is usually specified in terms of the data type of the column. so an offset spec of [5] means a 20-byte offset if the data type is a 32-bit integer, and a 40-byte offset for a double. if you want to specify a byte offset instead of one tied to the column data type, precede the offset value with 'B' — e.g. [B6] means a 6-byte offset, no matter what the column data type is 🎯
the [poff] is especially useful in conjunction with the pointer @ specification, since it lets the data element be stored anywhere in the allocated array. for example, a spec like "@I[2]" specifies the third (i.e., starting from 0) element in the array pointed to by the pointer value. a value of "@2I[4]" specifies the fifth and sixth values in the array. like, consider this spec:
typedef struct EvStruct{
short x[4], *atp;
} *Event, EventRec;
/* set up the (hardwired) columns */
FunColumnSelect( fun, sizeof(EventRec), NULL,
"2i", "2I ", "w", FUN_OFFSET(Event, x),
"2i2", "2I[2]", "w", FUN_OFFSET(Event, x),
"at2p", "@2I", "w", FUN_OFFSET(Event, atp),
"at2p4", "@2I[4]", "w", FUN_OFFSET(Event, atp),
"atp9", "@I[9]", "w", FUN_OFFSET(Event, atp),
"atb20", "@I[B20]", "w", FUN_OFFSET(Event, atb),
NULL);here we've specified the following columns:
2i: two short ints in an array which is stored as part of the record2i2: the 3rd and 4th elements of an array which is stored as part of the record- an array of at least 10 elements, not stored in the record but allocated elsewhere, and used by three different columns:
at2p: 2 short ints which are the first 2 elements of the allocated arrayat2p4: 2 short ints which are the 5th and 6th elements of the allocated arrayatp9: a short int which is the 10th element of the allocated array
atb20: a short int which is at byte offset 20 of another allocated array
and that's how you can specify several columns that all live in a single array. so iconic. NB: it is the programmer's responsibility to ensure that specification of a positive value for poff does not point past the end of valid data. that's on you, bestie 🫠
- read/write mode —
"r"means the column is read from an input file into user space byFunTableRowGet(), and"w"means the column is written to an output file. both can be specified at the same time 👯 - offset — the offset into the user data to store this column. typically the macro
FUN_OFFSET(recname, colname)is used to define the offset into a record structure. it's giving convenience again 🎀
when all column arguments have been specified, a final NULL argument must be added to signal the end of the column selection list. don't forget it or things get weird 🚨
as an alternative to the varargs FunColumnSelect() routine, there's also a non-varargs routine called FunColumnSelectArr(). the first three arguments (fun, size, plist) are the same as in FunColumnSelect(). instead of a variable argument list though, FunColumnSelectArr() takes 5 additional arguments. the first 4 array arguments contain the names, types, modes, and offsets, respectively, of the columns being selected. the final argument is the number of columns contained in those arrays. and heads up: it is the user's responsibility to free the string space allocated in these arrays 🧹
consider this example:
typedef struct evstruct{
int status;
float pi, pha, *phas;
double energy;
} *Ev, EvRec;
FunColumnSelect(fun, sizeof(EvRec), NULL,
"status", "J", "r", FUN_OFFSET(Ev, status),
"pi", "E", "r", FUN_OFFSET(Ev, pi),
"pha", "E", "r", FUN_OFFSET(Ev, pha),
"phas", "@9E", "r", FUN_OFFSET(Ev, phas),
NULL);each time a row is read into the Ev struct, the "status" column is converted to an int data type (regardless of its data type in the file) and stored in the status value of the struct. similarly, "pi" and "pha", and the phas vector, are all stored as floats. note that the "@" sign indicates that the "phas" vector is a pointer to a 9 element array, rather than an array allocated in the struct itself. and the row record can then be processed as required:
okay bestie, we're back in the FunColumnSelect() chat, and the next bit is literally about writable columns, merging, and struct-of-arrays vibes. no cap. 🫶
/* get rows -- let routine allocate the row array */
while( (ebuf = (Ev)FunTableRowGet(fun, NULL, MAXROW, NULL, &got)) ){
/* process all rows */
for(i=0; i<got; i++){
/* point to the i'th row */
ev = ebuf+i;
ev->pi = (ev->pi+.5);
ev->pha = (ev->pi-.5);
}so like, FunColumnSelect() can also be called to define "writable" columns, and that's how you generate a FITS Binary Table without even referencing any input columns. it's giving standalone main character energy. ✨ for example, this next snippet will generate a 4-column FITS binary table when FunTableRowPut() is used to write Ev records:
typedef struct evstruct{
int status;
float pi, pha
double energy;
} *Ev, EvRec;
FunColumnSelect(fun, sizeof(EvRec), NULL,
"status", "J", "w", FUN_OFFSET(Ev, status),
"pi", "E", "w", FUN_OFFSET(Ev, pi),
"pha", "E", "w", FUN_OFFSET(Ev, pha),
"energy", "D", "w", FUN_OFFSET(Ev, energy),
NULL);every single column here is declared write-only, so bestie, the column data is presumably being generated or read from some other source. iykyk. 💅
plus, FunColumnSelect() can be called to define both "readable" and "writable" columns at the same time. in that case, the "read" columns are associated with an input file, while the "write" columns are associated with the output file. and yes, columns can be specified as both "readable" and "writable" — in which case they are read from input and (possibly modified data values are) written to the output. the FunColumnSelect() call itself is made by passing the input Funtools handle, and it is assumed that the output file has been opened using this input handle as its Funtools reference handle. periodt.
consider this iconic example:
typedef struct evstruct{
int status;
float pi, pha, *phas;
double energy;
} *Ev, EvRec;
FunColumnSelect(fun, sizeof(EvRec), NULL,
"status", "J", "r", FUN_OFFSET(Ev, status),
"pi", "E", "rw", FUN_OFFSET(Ev, pi),
"pha", "E", "rw", FUN_OFFSET(Ev, pha),
"phas", "@9E", "rw", FUN_OFFSET(Ev, phas),
"energy", "D", "w", FUN_OFFSET(Ev, energy),
NULL);just like in the "read" example above, every time a row is read into the Ev struct, the "status" column is converted to an int data type (no matter what its data type is in the file) and stored in the status value of the struct. same vibe for "pi" and "pha", and the phas vector — they're all stored as floats. and since "pi", "pha", and "phas" are declared as "writable" as well as "readable", they also get written to the output file. but bestie, note that "status" is declared "readable" only, so it will not be written to an output file. finally, the "energy" column is declared "writable" only, meaning it will not be read from the input file. in this case, it can be assumed that "energy" will be calculated in the program before being output along with the other values. slay. 💖
in these simple cases, only the columns specified as "writable" will be output using FunTableRowPut(). however, it often is the case that you want to merge the user columns back in with the input columns, even when not all of the input column names are explicitly read or even known. for this important case, the merge=[type] keyword is provided in the plist string. it's giving merge era. 🌸
the merge=[type] keyword tells Funtools to merge the columns from the input file with user columns on output. it's normally used when an input and output file are opened and the input file provides the Funtools reference handle for the output file. in this case, each time FunTableRowGet() is called, the raw input rows are saved in a special buffer. if FunTableRowPut() then is called (before another call to FunTableRowGet()), the contents of the raw input rows are merged with the user rows according to the value of type as follows:
update— add new user columns, and update value of existing ones (maintaining the input data type) 🆙replace— add new user columns, and replace the data type and value of existing ones. (Note that if tlmin/tlmax values are not specified in the replacing column, but are specified in the original column being replaced, then the original tlmin/tlmax values are used in the replacing column.) 🔁append— only add new columns, do not "replace" or "update" existing ones ➕
consider the example above. if merge=update is specified in the plist string, then "energy" will be added to the input columns, and the values of "pi", "pha", and "phas" will be taken from the user space (i.e., the values will be updated from the original values, if they were changed by the program). the data type for "pi", "pha", and "phas" will be the same as in the original file. if merge=replace is specified, both the data type and value of these three input columns will be changed to the data type and value in the user structure. if merge=append is specified, none of these three columns will be updated, and only the "energy" column will be added. note that in all cases, "status" will be written from the input data, not from the user record, since it was specified as read-only. no cap. 💅
standard applications will call FunColumnSelect() to define user columns. however, if this routine is not called, the default behavior is to transfer all input columns into user space. for this purpose a default record structure is defined such that each data element is properly aligned on a valid data type boundary. this mechanism is used by programs such as fundisp and funtable to process columns without needing to know the specific names of those columns. it is not anticipated that users will need such capabilities (contact us if you do!) — not me putting that in a manpage but go off. 😭
by default, FunColumnSelect() reads/writes rows to/from an "array of structs", where each struct contains the column values for a single row of the table. this means that the returned values for a given column are not contiguous. you can set up the IO to return a "struct of arrays" so that each of the returned columns are contiguous by specifying org=structofarrays (abbreviation: org=soa) in the plist. (the default case is org=arrayofstructs or org=aos.) lowkey a vibe switch. ✨
for example, the default setup to retrieve rows from a table would be to define a record structure for a single event and then call FunColumnSelect() as follows:
typedef struct evstruct{
short region;
double x, y;
int pi, pha;
double time;
} *Ev, EvRec;
got = FunColumnSelect(fun, sizeof(EvRec), NULL,
"x", "D:10:10", mode, FUN_OFFSET(Ev, x),
"y", "D:10:10", mode, FUN_OFFSET(Ev, y),
"pi", "J", mode, FUN_OFFSET(Ev, pi),
"pha", "J", mode, FUN_OFFSET(Ev, pha),
"time", "1D", mode, FUN_OFFSET(Ev, time),
NULL);subsequently, each call to FunTableRowGet() will return an array of structs, one for each returned row. if instead you wanted to read columns into contiguous arrays, you specify org=soa:
typedef struct aevstruct{
short region[MAXROW];
double x[MAXROW], y[MAXROW];
int pi[MAXROW], pha[MAXROW];
double time[MAXROW];
} *AEv, AEvRec;
got = FunColumnSelect(fun, sizeof(AEvRec), "org=soa",
"x", "D:10:10", mode, FUN_OFFSET(AEv, x),
"y", "D:10:10", mode, FUN_OFFSET(AEv, y),
"pi", "J", mode, FUN_OFFSET(AEv, pi),
"pha", "J", mode, FUN_OFFSET(AEv, pha),
"time", "1D", mode, FUN_OFFSET(AEv, time),
NULL);note that the only modification to the call is in the plist string. that's it. iconic. 💅
of course, instead of using statically allocated arrays, you also can specify dynamically allocated pointers:
/* pointers to arrays of columns (used in struct of arrays) */
typedef struct pevstruct{
short *region;
double *x, *y;
int *pi, *pha;
double *time;
} *PEv, PEvRec;
got = FunColumnSelect(fun, sizeof(PEvRec), "org=structofarrays",
"$region", "@I", mode, FUN_OFFSET(PEv, region),
"x", "@D:10:10", mode, FUN_OFFSET(PEv, x),
"y", "@D:10:10", mode, FUN_OFFSET(PEv, y),
"pi", "@J", mode, FUN_OFFSET(PEv, pi),
"pha", "@J", mode, FUN_OFFSET(PEv, pha),
"time", "@1D", mode, FUN_OFFSET(PEv, time),
NULL);here, the actual storage space is either allocated by the user or by the FunColumnSelect() call). (yes, that closing paren is just vibing there. 🫶)
in all of the above cases, the same call is made to retrieve rows, e.g.:
buf = (void *)FunTableRowGet(fun, NULL, MAXROW, NULL, &got);however, the individual data elements are accessed differently. for the default case of an "array of structs", the individual row records are accessed using:
for(i=0; i<got; i++){
ev = (Ev)buf+i;
fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n",
ev->x, ev->y, ev->pi, ev->pha, ev->dx, ev->dy, ev->time);
}for a struct of arrays or a struct of array pointers, we have a single struct through which we access individual columns and rows using:
aev = (AEv)buf;
for(i=0; i<got; i++){
fprintf(stdout, "%.2f\t%.2f\t%d\t%d\t%.4f\t%.4f\t%21.8f\n",
aev->x[i], aev->y[i], aev->pi[i], aev->pha[i],
aev->dx[i], aev->dy[i], aev->time[i]);
}support for struct of arrays in the FunTableRowPut() call is handled analogously. fr fr. 🦋
see the evread example code and evmerge example code for working examples of how FunColumnSelect() is used. go look, bestie. 👑
✨ SEE ALSO ✨
see funtools(7) for a list of Funtools help pages. that's the whole directory of vibes. 📚
💖 and that's the tea on funcolumnselect(3) — version 1.4.5, dated April 14, 2011. periodt. go slay. 🌈
📦 where this page comes from
- debian bookworm: libfuntools-dev 1.4.8-1 · GPL
- debian trixie: libfuntools-dev 1.4.8-1.1+b2 · GPL
- ubuntu jammy: libfuntools-dev 1.4.8-1 · GPL
- ubuntu noble: libfuntools-dev 1.4.8-1.1build2 · GPL
- ubuntu questing: libfuntools-dev 1.4.8-2
- ubuntu resolute: libfuntools-dev 1.4.8-2build1 · GPL
- ubuntu stonking: libfuntools-dev 1.4.8-2build1 · GPL
- 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms