STILTS-TSKYMAP(1)

📦
version 3.5.4 · 5 versions
, in ubuntu stonking

🦋 STILTS-TSKYMAP 🦋

okay bestie, hi 💖 welcome to stilts-tskymap, page 1, from the Stilts commands manual. she Calculates sky density maps and honestly? she's iconic for it 👑

✨ NAME ✨

stilts-tskymap — Calculates sky density maps.

that's the whole job description, no notes, she ate 💅

🎀 SYNOPSIS 🎀

stilts tskymap [ifmt=<in-format>] [istream=true|false] [in=<table>]
               [icmd=<cmds>] [ocmd=<cmds>]
               [omode=out|meta|stats|count|checksum|cgi|discard|topcat|samp|tosql|gui]
               [out=<out-table>] [ofmt=<out-format>] [lon=<expr/deg>]
               [lat=<expr/deg>]
               [tiling=hpx<K>|healpixnest<K>|healpixring<K>|htm<K>]
               [count=true|false] [cols=<expr>[;<combiner>[;<name>]]
               ...] [combine=sum|sum-per-unit|count|count-per-
               unit|mean|median|Q1|Q3|min|max|stdev|stdev_pop|hit]
               [perunit=steradian|degree2|arcmin2|arcsec2|mas2|uas2]
               [complete=true|false]
               [runner=sequential|parallel|parallel<n>|partest]

💫 DESCRIPTION 💫

ngl, here's the vibe: tskymap calculates a weighted density map — or, if you wanna say it the other way, a histogram — up on the sky, using columns from an input table 🌈

The sky gets sliced up into some discrete set of tiles according to whatever tessellation scheme you picked (currently HEALPix or HTM are supported, that's the roster), and the quantities you asked for get aggregated into bins that correspond to those tiles. So the output table has a column giving the pixel index that identifies each tile, plus one or more other columns, each one being an aggregation of a quantity from the input table 💖

By default, the number of rows from the input table that landed inside each tile gets included as the first column of the output table. But if you specify the cols and combine parameters, you can stack on more columns giving the sum, mean, median or other statistics of input table columns or expressions too ✨

Once you have that output table, you can do stuff like plot it with plot2sky's healpix layer type (though an alternative is to just do that plot directly with a skydensity layer). Either way, cute 🎀

If we're talking HEALPix tiling specifically, the result can also be written out to a FITS file suitable for external applications that understand the semi-standard FITS-Healpix convention. Note — and this bit matters — in that case, for maximum compatibility, the fits-healpix output format should in general be used 👑

Also go say hi to tgridmap, which does the exact same thing but for N-dimensional grid geometry 🫶

🌸 OPTIONS 🌸

  • ifmt=<in-format> — Says what format the input table (the one in parameter in) is in. The known formats are all listed in SUN/256. Use this flag when you already know what format your table is. If it's set to the special value (auto) — which is the default — then the program will attempt to figure out the format all by itself 🙈 It can't always get this right though, and when it can't, the program exits with an error explaining which formats it tried. Oh, and heads up: this parameter is ignored for scheme-specified tables.
  • istream=true|false — If you set it true, the input table from the in parameter gets read as a stream. You have to give the ifmt parameter if you do this. Depending on what operations you asked for and what processing mode you're in, this may make the read fail (sometimes the table genuinely has to be read more than once, sorry bestie 😭). It's not normally necessary to set this flag — in most cases the data streams automatically when that's the best thing to do. That said, it can sometimes mean less resource usage when you're processing large files in certain formats (VOTable, for instance). This parameter is ignored for scheme-specified tables.
  • in=<table> — Where the input table lives. It can take any one of these forms:
    • A filename.
    • A URL.
    • The special value -, which means standard input. In that case the input format must be given explicitly with the ifmt parameter. Note that not all formats can be streamed this way.
    • A scheme specification shaped like :<scheme-name>:<scheme-args>.
    • A system command line with either a < character at the start, or a | character at the end ("<syscmd" or "syscmd|"). This runs the given pipeline and reads from its standard output. It'll probably only work on unix-like systems 🐧

In any case, compressed data in one of the supported compression formats (gzip, Unix compress or bzip2) gets decompressed transparently ✨

  • icmd=<cmds> — Processing to be performed on the input table (the one in parameter in), before any other processing has happened. The value is one or more of the filter commands described in SUN/256. If you give more than one, they have to be separated by semicolon characters (;). You can repeat this parameter multiple times on the same command line to build up a list of processing steps, and the sequence of commands you give defines the processing pipeline that gets performed on the table 💅

Commands can alternatively be supplied in an external file using the indirection character @. So a value of "@filename" makes the file filename get read for a list of filter commands to execute. Commands in that file may be separated by newline characters and/or semicolons, and lines that are blank or that start with a # character are ignored. A backslash character '\fR' at the end of a line joins it with the following line.

  • ocmd=<cmds> — Processing to be performed on the output table, after all other processing has taken place. The value is one or more of the filter commands described in SUN/256. If you give more than one, they have to be separated by semicolon characters (;). You can repeat this parameter multiple times on the same command line to build up a list of processing steps, and the sequence of commands you give defines the processing pipeline that gets performed on the table 💖

Commands can alternatively be supplied in an external file using the indirection character @. So a value of "@filename" makes the file filename get read for a list of filter commands to execute. Commands in that file may be separated by newline characters and/or semicolons, and lines that are blank or that start with a # character are ignored. A backslash character '\fR' at the end of a line joins it with the following line.

  • omode=out|meta|stats|count|checksum|cgi|discard|topcat|samp|tosql|gui — The mode the result table gets output in. Default is out, which means the result gets written as a new table to disk or wherever, as decided by the out and ofmt parameters. But there are other possibilities — they line up with other uses a table can be put to besides outputting it, like displaying metadata, calculating statistics, or populating a table in an SQL database. For some values of this parameter you also need extra parameters (<mode-args>) to nail down the exact behaviour.

The possible values are:

  • out
  • meta
  • stats
  • count
  • checksum
  • cgi
  • discard
  • topcat
  • samp
  • tosql
  • gui

Use the help=omode flag or see SUN/256 for more info 🫶

  • out=<out-table> — Where the output table goes. Usually a filename to write to. If it's the special value - (which is the default), the output table gets written to standard output.
bestie, you only give this parameter if omode has its default value of "out" ⚠️
  • ofmt=<out-format> — What format the output table gets written in (one of the ones in SUN/256 — matching is case-insensitive and you can get away with just the first few letters). If it's the special value (auto) (the default), then the output filename gets examined to try and guess what sort of file is wanted, usually by peeking at the extension. If it's not obvious from the filename what output format is intended, you get an error 💀
same deal: only give this parameter if omode has its default value of "out" ⚠️
  • lon=<expr/deg> — Longitude in degrees for the position of each row in the input table. This can just be a column name, or it can be an algebraic expression, as explained in SUN/256. The sky system used here is what determines the grid the output map gets built on 🌐
  • lat=<expr/deg> — Latitude in degrees for the position of each row in the input table. Again, this can just be a column name, or it can be an algebraic expression, as explained in SUN/256. The sky system used here is what determines the grid the output map gets built on 🌐
  • tiling=hpx<K>|healpixnest<K>|healpixring<K>|htm<K> — Describes the sky tiling scheme in use. You can pick one of these:
    • hpxK: alias for healpixnestK
    • healpixnestK: HEALPix using the Nest scheme at order K
    • healpixringK: HEALPix using the Ring scheme at order K
    • htmK: Hierarchical Triangular Mesh at level K

So for example, hpx5 and healpixnest5 would both mean the HEALPix NEST tiling scheme at order 5 ✨

At level K, there are 12*4^K HEALPix pixels, or 8*4^K HTM pixels on the sky. More information about these tiling schemes can be found at the HEALPix and HTM web sites 🌈

  • count=true|false — Controls whether a COUNT column gets added to the output table alongside any other columns you may have requested. If it's included, it reports the number of rows from the input table that fell within the corresponding bin 🔢
  • cols=<expr>[;<combiner>[;<name>]] ... — Defines the quantities to be calculated. The value is a space-separated list of items, one for each aggregated column in the output table.

Each item is made up of one, two or three tokens, separated by semicolon (;) characters:

  • <expr>: (required) column name or expression using the expression language, for the quantity to be aggregated.

bestie that list is not done, there's more of this coming in the next part ✨

  • <combiner> — (optional) which combination method to use, taking the exact same options as the combine parameter. Leave it out and the value you set for that parameter gets used instead. ✨
    • <name> — (optional) the name of the output column. Omit it and the <expr> value gets used (maybe a little sanitised, ngl). 💅

Most of the time you can lowkey just hand over a space-separated list of input table column names here and that's it, bestie. But sometimes you need the extra syntax — like when you've gotta calculate both a sum and a mean of the same input column. That's the moment the fancy form earns its keep. 🫶

  • combine=sum|sum-per-unit|count|count-per-unit|mean|median|Q1|Q3|min|max|stdev|stdev_pop|hit — sets the default way that all the values landing in the same density map bin get squished together into the one value that bin ends up with. Here's the whole menu:
    • sum — the sum of all the combined values per bin 🌈
    • sum-per-unit — the sum of all the combined values per unit of bin size
    • count — the number of non-blank values per bin (the weight is ignored, she's not invited 💀)
    • count-per-unit — the number of non-blank values per unit of bin size (again, weight is ignored)
    • mean — the mean of the combined values
    • median — the median
    • Q1 — first quartile
    • Q3 — third quartile
    • min — the minimum of all the combined values
    • max — the maximum of all the combined values
    • stdev — the sample standard deviation of the combined values
    • stdev_pop — the population standard deviation of the combined values
    • hit — 1 if any values are present, NaN otherwise (weight ignored, obviously 🎀)
    • Q.nnn — quantile nnn (so like, Q.05 is the fifth percentile, periodt)

For the density-like ones (count-per-unit, sum-per-unit) the scaling also gets nudged by the perunit parameter. 👑

Heads up bestie: this value can be overridden on a per-column basis by the cols parameter. So your global vibe can be quietly changed locally, iykyk.
  • perunit=steradian|degree2|arcmin2|arcsec2|mas2|uas2 — picks the unit of sky area used to scale density-like combinations (think combine=count-per-unit or sum-per-unit). If the combination mode is doing values per unit area, this is what configures the area scale in question. For the non-density-like modes (like combine=sum or mean) it literally does nothing, no cap. 🦋

The options are:

  • steradian — steradian ✨
  • degree2 — square degree
  • arcmin2 — square arcminute
  • arcsec2 — square arcsecond
  • mas2 — square milliarcsec
  • uas2 — square microarcsec
  • complete=true|false — decides whether the output table has a row for every pixel in the tiling, or only the rows for pixels that some of the input data actually fell into. 🧚

This one affects performance as well as output, so listen up. If you already know most pixels on the sky are gonna be covered, setting it true is probably the move. And if you know only a small patch of sky is getting covered, false is the better call. Choose wisely 💜

  • runner=sequential|parallel|parallel<n>|partest — chooses the threading implementation, i.e. how much of the processing actually happens in parallel. Right now the options are:
    • sequential — runs using only a single thread 🐌
    • parallel — runs using multiple threads for large tables, with the parallelism given by the number of available processors
    • parallel<n> — runs using multiple threads for large tables, with the parallelism given by the value n you supply
    • partest — runs using multiple threads even when tables are small (only intended for testing purposes, don't get comfy with it 🙈)

Going parallel can speed things up considerably, bestie. But depending on the I/O operations required, it can also slow you all the way down by disrupting patterns of disk access. 😭 If a file's content lives on a solid state disk, or is already in cache — say, because a similar command ran recently — then parallel will probably be faster. But if the data is being read directly off a spinning disk, like when the file is too large to fit in RAM, then sequential or parallel<n> with a small n may be faster. It's giving ✨situational✨.

This parameter should make only very tiny differences to the output table. If you notice significant discrepancies, please report them. 🔥

🌸 SEE ALSO 🌸

If the package stilts-doc is installed, the full documentation SUN/256 is available in HTML format at:

file:///usr/share/doc/stilts/sun256/index.html

👑 VERSION 👑

STILTS version 3.5.4-debian

This is the Debian version of Stilts, which lacks the support of some file formats and network protocols. For differences, see:

file:///usr/share/doc/stilts/README.Debian

🎀 AUTHOR 🎀

Mark Taylor (Bristol University)

Okay bestie that's the whole page, we ate and left no crumbs 💖 It's the STILTS 3.5.4-debian flavour, dated Mar 2017, and yes — it really does call itself STILTS-TSKYMAP(1). Go map some sky, you main character. ⭐🌈

📦 where this page comes from