STILTS-TSKYMATCH2(1)

📦
version 3.5.4 · 5 versions
, in ubuntu stonking

💫 STILTS-TSKYMATCH2 💫

okay bestie, welcome to the STILTS-TSKYMATCH2 page from the Stilts commands manual 🌸 this one is a whole vibe: it crossmatches two tables on sky position, and it is genuinely so good at it. let's go 🫶

✨ NAME ✨

stilts-tskymatch2 — Crossmatches 2 tables on sky position 🎯✨

🦋 SYNOPSIS 🦋

stilts tskymatch2 [in1=<table1>] [ifmt1=<in-format>] [in2=<table2>]
                  [ifmt2=<in-format>]
                  [omode=out|meta|stats|count|checksum|cgi|discard|topcat|samp|tosql|gui]
                  [out=<out-table>] [ofmt=<out-format>] [ra1=<expr>]
                  [dec1=<expr>] [ra2=<expr>] [dec2=<expr>]
                  [error=<value/arcsec>] [tuning=<healpix-k>]
                  [join=1and2|1or2|all1|all2|1not2|2not1|1xor2]
                  [find=all|best|best1|best2]
                  [runner=parallel|parallel<n>|parallel-
                  all|sequential|classic|partest]

💖 DESCRIPTION 💖

So tskymatch2 does a crossmatch of two tables based on how close their sky positions are. You tell it the columns (or expressions) that give right ascension and declination in degrees for each input table, plus a maximum permissible separation in arcseconds, and it hands you the resulting joined table. no cap, that's the whole thing 💅

Now, if you leave the RA and Dec expressions blank, it will try to figure out which columns you mean by using column Unified Content Descriptors (UCDs) or names. Here's the tea ☕:

  • First it hunts for columns with appropriate UCD1 or UCD1+ values: POS_EQ_RA, POS_EQ_RA_MAIN, pos.eq.ra or pos.eq.ra;meta.main — and their declination equivalents.
  • If it can't find those, it goes looking for columns named something like "RA" or "RA2000" 👀
  • If either turns up, the column units get consulted, and radian->degree conversions are performed if necessary. (If no unit value is given, degrees are assumed.)
  • If it genuinely can't find anything likely, the command will fail with an error message 😭
⚠️ Heads up bestie: this search logic is meant as a convenience only. It's kinda ad hoc and subject to change. So if you want to be 100% sure the correct angle values are being used, specify the ra and dec position parameters explicitly. periodt.

Also, real talk: tskymatch2 is just a cut-down version of the more general matching task tmatch2, provided for convenience. So if you want more match options or just generally more configurability, you can probably find it by using tmatch2 🌈

💅 OPTIONS 💅

  • in1=<table1> — where your first input table lives. It can be any of these:
    • A filename.
    • A URL.
    • The special value "-", meaning standard input. In that case you must give the input format explicitly using the ifmt1 parameter. Not every format can be streamed this way, so don't come for me if it doesn't work 🙈
    • A scheme specification of the form :<scheme-name>:<scheme-args>.
    • A system command line with either a "<" character at the start, or a "|" character at the end ("<syscmd" or "syscmd|"). This runs the given pipeline and reads from its standard output. This will probably only work on unix-like systems.

In any case, compressed data in one of the supported compression formats (gzip, Unix compress or bzip2) will be decompressed transparently 🎀

  • ifmt1=<in-format> — the format of the first input table (the one from in1). The known formats are listed in SUN/256. Use this flag when you already know what format your table is in. If it has the special value (auto) — which is the default — then it'll try to detect the format automatically. It can't always get that right, though, in which case the program exits with an error explaining which formats it tried 😤 This parameter is ignored for scheme-specified tables.
  • in2=<table2> — where your second input table lives. Same possibilities as in1:
    • A filename.
    • A URL.
    • The special value "-", meaning standard input. In that case you must give the input format explicitly using the ifmt2 parameter. Again: not all formats can be streamed this way 🫠
    • A scheme specification of the form :<scheme-name>:<scheme-args>.
    • A system command line with either a "<" character at the start, or a "|" character at the end ("<syscmd" or "syscmd|"). This runs the given pipeline and reads from its standard output. This will probably only work on unix-like systems.

And yep — compressed data in one of the supported compression formats (gzip, Unix compress or bzip2) will be decompressed transparently ✨

  • ifmt2=<in-format> — the format of the second input table (the one from in2). The known formats are listed in SUN/256. Use this flag when you know what format your table is in. If it has the special value (auto) (the default), it'll attempt automatic format detection. That can't always be done correctly, and when it can't, the program exits with an error telling you which formats it attempted 💀 This parameter is ignored for scheme-specified tables.
  • omode=out|meta|stats|count|checksum|cgi|discard|topcat|samp|tosql|gui — the mode in which the result table gets output. The default mode is out, meaning the result is written as a new table to disk or wherever, as determined by the out and ofmt parameters. But there are other possibilities, corresponding to other uses a table can be put to besides outputting it — like displaying metadata, calculating statistics, or populating a table in an SQL database. For some values of this parameter, additional parameters (<mode-args>) are required to determine the exact behaviour 👑

Possible values:

  • out
  • meta
  • stats
  • count
  • checksum
  • cgi
  • discard
  • topcat
  • samp
  • tosql
  • gui

Use the help=omode flag or see SUN/256 for more info 💖

  • out=<out-table> — where the output table goes. Usually a filename to write to. If it equals the special value "-" (which is the default), the output table is written to standard output.
This parameter must only be given if omode has its default value of "out". 🚨
  • ofmt=<out-format> — the format the output table gets written in (one of the ones in SUN/256 — matching is case-insensitive and you can use just the first few letters, slay). If it has the special value "(auto)" (the default), the output filename gets examined to guess what sort of file is required, usually by looking at the extension. If it's not obvious from the filename what output format is intended, you get an error 😵
This parameter must only be given if omode has its default value of "out". 🚨
  • ra1=<expr> — right ascension in degrees for the position of each row of table 1. This can just be a column name, or it can be an algebraic expression calculated from columns, as explained in SUN/256. If left blank, an attempt is made to guess from UCDs, column names and unit annotations what expression to use 🔭
  • dec1=<expr> — declination in degrees for the position of each row of table 1. Can be a plain column name or an algebraic expression calculated from columns, as explained in SUN/256. If left blank, it tries to guess from UCDs, column names and unit annotations what expression to use 🌌
  • ra2=<expr> — right ascension in degrees for the position of each row of table 2. Column name or algebraic expression calculated from columns, per SUN/256. If left blank, an attempt is made to guess from UCDs, column names and unit annotations what expression to use 💫
  • dec2=<expr> — declination in degrees for the position of each row of table 2. Again: column name or algebraic expression calculated from columns, as explained in SUN/256. If left blank, it guesses from UCDs, column names and unit annotations what expression to use 🦋
  • error=<value/arcsec> — the maximum separation permitted between two objects for them to count as a match. Units are arc seconds 📏
  • tuning=<healpix-k> — a tuning parameter that controls the pixel size used when binning the rows. The legal range is from 0 (corresponding to pixel size of about 60 degrees) to 20 (about 0.2 arcsec). Heads up: the value of this parameter will not affect the result, but it may affect the performance in terms of CPU and memory resources required. A default value gets chosen based on the size of the error parameter, but it may be possible to improve performance by adjusting that default. You can see the value used by examining the progress output 👀 If your match is taking a long time or is failing from lack of memory, it may be worth trying different values for this parameter. tune responsibly bestie 💅
  • join=1and2|1or2|all1|all2|1not2|2not1|1xor2 — determines which rows are included in the output table. The matching algorithm figures out which of the rows from the first table correspond to which rows from the second; this parameter decides what to do with that information. Probs the most obvious move is writing out a table containing only rows that correspond to a row in both input tables. But you might also want to see the unmatched rows from one or both input tables, or rows present in one table but unmatched in the other, or other possibilities. The options are:
    • 1and2: An output row for each row represented in both input tables (INNER JOIN)
    • 1or2: An output row for each row represented in either or both of the input tables (FULL OUTER JOIN)
    • all1: An output row for each matched or unmatched row in table 1 (LEFT OUTER JOIN)
    • all2: An output row for each matched or unmatched row in table 2 (RIGHT OUTER JOIN)
    • 1not2: An output row only for rows which appear in the first table but are not matched in the second table
  • 2not1 — an output row only for rows that show up in the second table but get no match in the first one 🫶
  • 1xor2 — an output row only for rows that are represented in one of the input tables but not the other. it's giving exclusive, periodt 💅

find=all|best|best1|best2 🎀

okay so this one decides what happens when a row in one table can be matched by more than one row in the other table. like, the popular kid situation. your options:

  • all — all matches, bestie. every single match between the two tables lands in the result. rows from both input tables may show up multiple times in the result. no one gets left out 💖
  • best — best match, symmetric ✨. the best pairs get picked in a way that treats the two tables symmetrically. any input row that appears in one result pair is disqualified from appearing in any other result pair, so each row from both input tables appears in at most one row of the result. monogamy, but for tables 💅
  • best1 — best match for each Table 1 row 🌸. for every row in table 1, only the best match from table 2 makes it into the result. each row from table 1 appears a maximum of once in the result, but rows from table 2 may appear multiple times. table 1 is the main character fr fr 👑
  • best2 — best match for each Table 2 row 🦋. for every row in table 2, only the best match from table 1 makes it into the result. each row from table 2 appears a maximum of once in the result, but rows from table 1 may appear multiple times. now table 2 is the main character 💜

lowkey the differences between best, best1 and best2 are subtle, and i need you to hear me on this 😭 In cases where it's obvious which object in each table is the best match for which object in the other, choosing between these options will not affect the result. iconic, no drama. However, in crowded fields — where the distance between objects within one or both tables is typically similar to or smaller than the specified match radius — it will make a difference. In that case one of the asymmetric options (best1 or best2) is usually more appropriate than best, but you'll have to think about which of them suits your requirements. it's giving ✨soul-searching✨. The performance (time and memory usage) of the match may also differ between these options, especially if one table is much bigger than the other. so like, not just vibes, but actual resources 🫠

runner=parallel|parallel&lt;n&gt;|parallel-all|sequential|classic|partest 🧚

this one picks the threading implementation. currently on the menu:

  • parallel — uses the multithreaded implementation for large tables, with default parallelism, which is the smaller of 6 and the number of available processors 🍓
  • parallel<n> — uses the multithreaded implementation for large tables, with parallelism given by the supplied value <n> ⭐
  • parallel-all — uses the multithreaded implementation for large tables, with a parallelism given by the number of available processors 🌈
  • sequential — uses the multithreaded implementation but with only a single thread 🙈
  • classic — uses the legacy sequential implementation 💀
  • partest — uses the multithreaded implementation even when tables are small 🔥

The parallel* options should normally run faster than sequential or classic (which are provided mainly for testing purposes), at least for large matches and where multiple processing cores are available. slay, use the cores bestie 💅

The default value parallel is currently limited to a parallelism of 6 since larger values yield diminishing returns given that some parts of the matching algorithms run sequentially (Amdahl's Law), and using too many threads can sometimes end up doing more work or impacting on other operations on the same machine. yeah. Amdahl's Law said what it said 🎀 But you can experiment with other concurrencies, e.g. parallel16 to run on 16 cores (if available) or parallel-all to run on all available cores. go off, honestly 👑

The value of this parameter should make no difference to the matching results. If you notice any discrepancies please report them. not me begging you to file a bug 🥺

SEE ALSO 🌸

If the package stilts-doc is installed, the full documentation SUN/256 is available in HTML format: file:///usr/share/doc/stilts/sun256/index.html ✨

VERSION 💖

STILTS version 3.5.4-debian

This is the Debian version of Stilts, which lack the support of some file formats and network protocols. For differences see file:///usr/share/doc/stilts/README.Debian 🫶

AUTHOR 👑

Mark Taylor (Bristol University)

and that's the whole page bestie, we did it 🎉 this lil manual page is from STILTS version 3.5.4-debian and it's dated Mar 2017 — yes, she's a vintage icon, and she still slays 💅✨ bye ily 🫶

📦 where this page comes from