RFC 4047

MIME Sub-type Registrations for Flexible Image Transport System (FITS)

๐Ÿ“… published April 2005, from ietf

MIME Sub-type Registrations for Flexible Image Transport System (FITS) ๐ŸŽ€

okay bestie, this is RFC 4047 and it's giving astronomy meets internet ๐Ÿ”ญโœจ It's an Informational doc from April 2005, written by S. Allen (UCO/Lick Observatory) and D. Wells (National Radio Astronomy Observatory). It isn't a standard, but it's the official paperwork for two MIME types for FITS files. Its title block doesn't list anything it obsoletes or updates, so it's a fresh start ๐Ÿ’…

Status of This Memo ๐Ÿ“

Real talk: this memo gives information to the Internet community. It is not an Internet standard of any kind ๐Ÿ™ˆ Distribution is totally unlimited, so share it with everyone ๐Ÿซถ

Copyright (C) The Internet Society (2005). Legal stuff, but it's important ๐Ÿ’œ

Abstract ๐ŸŒธ

This doc is all about registering MIME sub-types so the international astronomy crowd can swap Flexible Image Transport System (FITS) files. The encoding is defined by the published FITS standard documents. FITS has been around since 1979, and basically all astronomical observation data gets shared this way. Iconic longevity ๐Ÿ‘‘

Table of Contents ๐Ÿ“š

   1.  Introduction..................................................  2
   2.  Conventions Used in this Document.............................  2
   3.  Overview......................................................  2
   4.  FITS Definition...............................................  3
       4.1.  FITS Structure..........................................  3
       4.2.  History of FITS Features................................  5
       4.3.  Stability of the FITS definition........................  6
       4.4.  Portability of FITS files...............................  7
       4.5.  Application Programming Interfaces to FITS..............  7
       4.6.  FITS File Conformance Testing...........................  8
       4.7.  Archives That Distribute FITS Files.....................  8
   5.  IANA Considerations...........................................  9
       5.1.  Registration of application/fits........................ 10
       5.2.  Registration of image/fits.............................. 14
   6.  References.................................................... 19
       6.1.  Normative References.................................... 19
       6.2.  Informative References.................................. 20
   7.  Security Considerations....................................... 21
   8.  Contributors.................................................. 21
   9.  Acknowledgements.............................................. 22
   Authors' Addresses................................................ 22
   Full Copyright Statement.......................................... 23

1. Introduction โœจ

So FITS is a file format that was built so astronomers could swap image data between observatories ๐Ÿ”ญ It carries arrays and tables of data, plus keyword/value pairs of metadata. The standards get approved by the International Astronomical Union (IAU, http://www.iau.org/) and published in refereed journals. Big credibility energy ๐Ÿ˜ค

Before HTTP even existed, astronomers were already using the Internet to trade FITS files. Multiple unofficial media types for FITS files [ASU] popped up shortly after the WWW started and have stuck around. Right now (2005), the international astronomy community is working on a bunch of cooperative projects (e.g., [IVOA], [NVO], [AstroGrid], [AVO]) to build web services that serve astronomical data ๐ŸŒˆ Passing FITS files around is a core part of the prototypes for these services [SIAP]. The community needs to agree on one set of media types for FITS files so all its services can work together. Teamwork makes the dream work ๐Ÿซถ

Here's the simplest way to think about it: FITS carries astronomical image data as a raster, along with coordinate info and other standard and locally defined metadata. In that role it's pretty much like the famous TIFF format [TIFF] plus the GeoTIFF tags [GeoTIFF]. The difference? FITS can describe way more than 2-dimensional rasters ๐Ÿงš The FITS community agreed that two media types are needed: one for images and one for all other cases. Clean split, no notes ๐Ÿ’–

2. Conventions Used in this Document ๐ŸŽ€

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC-2119 [Require]. These are the official key words, so when they show up in caps they carry real weight ๐Ÿ”ฅ

3. Overview ๐Ÿฆ‹

This doc is about registering the MIME media sub-types "application/fits" and "image/fits" โœจ

Back in 1988, the International Astronomical Union set up the FITS Working Group (IAUFWG) to look after how the FITS data format evolves ๐ŸŒธ The IAUFWG approved sending this document in and registering these two MIME types. Official sign-off, iykyk ๐Ÿ’…

4. FITS Definition ๐Ÿ”ญ

FITS is defined by a document that the International Astronomical Union (IAU) approved and published in the journal Astronomy & Astrophysics [NOST]. When people want extra keywords used in FITS files, they propose them, and interested parties negotiate and review them through ad hoc committees of the FITS community. If national committees and the IAUFWG approve that extra keyword usage, the keywords become new reserved keywords in the FITS standard and get published in an ongoing series of papers (e.g., [WCS1, WCS2]) ๐ŸŒˆ Keyword drama, but make it academic ๐Ÿ™ˆ

You can grab copies of the standard documents from these sites:

http://fits.gsfc.nasa.gov/

http://archive.stsci.edu/fits/

http://www.cv.nrao.edu/fits/

http://heasarc.gsfc.nasa.gov/docs/heasarc/fits.html

This section gives a quick background on FITS structure and features, but for the complete technical details, you need the FITS standards documents directly ๐Ÿ“š๐Ÿ’œ

4.1. FITS Structure โœจ

okay so a FITS file is basically a stack of header and data units (HDUs), optionally followed by special records ๐Ÿ“ฆ. The whole thing is built out of blocks that are 2880 8-bit bytes long (23040 bits) ๐Ÿ’…. That size was picked because it divides evenly by the byte and word lengths of every known computer system, which is so smart ๐Ÿง . Every FITS file's length is a whole-number multiple of that block size, so no half-blocks allowed ๐Ÿ™…โ€โ™€๏ธ

Each FITS header is one or more 2880-byte blocks holding 36 80-character records (36*80=2880) ๐ŸŽ€. The records are ASCII keyword/value pairs plus optional comments. The character set is the 7-bit printing ASCII codes, and that includes the ASCII space. The control codes CR, LF, FF, TAB and NUL are not used in FITS headers, so they're banned from the party ๐Ÿšซ Keywords can be up to 8 characters long. Some keywords are mandatory and their meaning is rigidly prescribed (no vibes-based interpretation here ๐Ÿ˜ค). Those ones describe the structure and size of the data array that comes after. The standard also reserves other keywords for specific metadata, and any keyword that's neither mandatory nor reserved can be added with semantics defined by local conventions. Fun fact: some local conventions have since been adopted into standardized practice โœจ. The end of the header is marked by a block containing the keyword END ๐Ÿ

Here's a simple FITS header for a digital image. It only uses keywords from the initial FITS Agreement of March 1979 [FITS], and it's giving classic ๐Ÿฆ‹

            1         2         3         4         5         6
   1234567890123456789012345678901234567890123456789012345678901234567..

   SIMPLE  =                    T / file does conform to FITS standard
   BITPIX  =                   16 / 16-bit twos-complement pixel values
   NAXIS   =                    2 / 2-dimensional image
   NAXIS1  =                  512 / first axis length
   NAXIS2  =                  512 / second axis length
   COMMENT  -----------------------------------------------------
   COMMENT   FITS (Flexible Image Transport System) format is defined
   COMMENT   in 'Astronomy and Astrophysics', volume 376, page 359;
   COMMENT   bibcode: 2001A&A...376..359H
   COMMENT  -----------------------------------------------------
   ORIGIN  = 'Lick Observatory   ' /
   DATE    = '2003-11-22T05:23:45' / when this file was written
   END

Those 13 records get followed by 23 more 80-character records of ASCII spaces, which just pads the header out to the 2880-byte block size ๐Ÿ›

The first keyword/value pair, SIMPLE=T, is the signature of FITS, and every FITS file starts with those characters ๐ŸŒธ BITPIX is the second keyword in every FITS file, and its legal values are 8, 16, 32, -32 and -64. NAXIS is the third keyword in every FITS file, and its legal values are 0 to 999. NAXIS=0 is legal and means there's no image data matrix tied to the header, and in that case the NAXISi keywords must not be present. This value is common for files that get the media type "application/fits", which we'll cover in section 5.1 of this document ๐Ÿ“Ž NAXIS=2 is the value used most often for files that get the media type "image/fits", covered in section 5.2 of this document. (NAXIS=3 FITS images are also super common, just so you know ๐Ÿ‘€)

The COMMENT records get ignored by FITS reading software, but they're here to hold the exact journal citation for the FITS standard. That info matters a lot for archiving over decades ๐Ÿ“š The ORIGIN keyword is commonly used to name the institution where the file was made ๐Ÿ›๏ธ The DATE keyword gives the timestamp for the file, while DATE-OBS is the one for observation start times. Real headers usually have a whole bunch of extra keywords for metadata. A digital image header will often include keywords for the precise celestial coordinates of the pixels in the 2-D matrix, following conventions that became part of the FITS standard in 2002 (see [WCS2]) ๐ŸŒŒ And here's the wild part: the example above shows that, besides the COMMENT records, 60% of the bytes in keyword=value records in FITS headers are reserved for comments ๐Ÿ™ˆ Even though this simple example fits in one FITS block, multi-block headers are commonly interchanged, so don't be surprised ๐Ÿซถ

Each header is followed by a data unit made of zero or more 2880-byte blocks. Based on what the header keywords say, those blocks hold an N-dimensional data array, optionally followed by other small groups of array data. Most of the time the data array is either an N-dimensional array of image pixel values or a 2-dimensional array of tabular data ๐Ÿ“Š

After the HDUs, a FITS file may contain zero or more 2880-byte blocks of special records. The standard doesn't say anything about what's in them ๐Ÿคทโ€โ™€๏ธ (This special records convention isn't known to have been used for anything except prototyping new parts of the standard, like random groups and BINTABLE ๐Ÿงช)

The first HDU in a FITS file is the primary HDU (PHDU), and any HDU after that is an extension HDU (XHDU) ๐Ÿ‘‘ The PHDU's keyword content is distinct. A PHDU needs a slight alteration before it can become an XHDU, and most types of XHDU can't become a PHDU, so the roles are pretty locked in ๐Ÿ”’ A PHDU can have a data array with zero elements, which will often happen for FITS files meant to carry tables and multiple images. An XHDU may be one of several standard types, or it may be another conforming type. The standard XHDU types are "IMAGE" (an N-dimensional data array, similar in most respects to the PHDU), "TABLE" (a 2-dimensional table of ASCII character data), and "BINTABLE" (a 2-dimensional table of binary data whose elements may themselves be multi-dimensional arrays) ๐Ÿ’–

4.2. History of FITS Features

okay the history is kinda iconic ๐Ÿฅน In 1981 the original definition of FITS described a single HDU holding one multi-dimensional image array [FITS], like the sample header in section 4.1 shows. Later agreements kept the original HDU and keyword/value framework to extend FITS while keeping every existing file valid โœจ Now FITS has standard ways to describe multiple arrays of image data and/or multiple tables of numeric and character info. Main character energy ๐Ÿ‘‘

Brief highlights of the history of FITS

  • 1979: Initial FITS Agreement and first interchange of files
  • 1980: Random groups convention developed
  • 1981: Published original (single HDU) definition [FITS]
  • 1981: Published random groups definition [GROUPS]
  • 1982: Formally endorsed by the IAU
  • 1988: Defined rules for multiple HDUs [XTENSION]
  • 1988: FITS Working Group established by IAU [IAUFWG]
  • 1988: Extended to include ASCII tables [TABLE]
  • 1990: Extended to include IEEE floating-point data
  • 1994: Extended to multiple image arrays [IMAGE]
  • 1995: Extended to binary tables [BINTABLE]
  • 1997: Adopted a Y2K-compliant date format
  • 2001: Reiterated existing standard in one paper [NOST]
  • 2002: Approved conventions for world coordinates [WCS1, WCS2]

4.3. Stability of the FITS Definition

so after the IAU adopted FITS in 1982, the emerging FITS community realized there would be a tension ๐Ÿ˜ฌ They needed archival stability but also needed the format to evolve. To get both, they set up a controlled parliamentary process ๐Ÿ›๏ธ When the Generalized Extensions Agreement [XTENSION] was made (that's the meta-agreement controlling how FITS evolves), the community adopted the guiding principle "Once FITS, always FITS" ๐Ÿ”’ Under this rule, no change may be made to FITS that invalidates existing files. Changes to the FITS standard only happen after action by the IAU FITS Working Group (FWG). And the FWG only acts after approval from regional working groups that coordinate FITS activity around the world ๐ŸŒ

FITS is the archival format for image data and interferometric data from many ground-based observatories, and for all data from many spacecraft ๐Ÿ›ฐ๏ธ Lots of astrophysical archives store their data in FITS, and most astronomical catalog data has been transcribed into FITS files. The many terabytes sitting in these archives are a big reason the FITS standard stays so stable ๐Ÿ’œ

4.4. Portability of FITS files

Eric Greisen, one of the authors of the original document [FITS], tells the story of 1979 like this ๐Ÿฅบ

[t]he first FITS files were written by a PL/I program on an IBM 360 under OS/MFT (32-bit, twos-complement numbers and 8-bit EBCDIC characters) and were read by a Fortran program executing on a CDC 6400 under SCOPE (60-bit, ones-complement numbers and 6-bit "Display Code" characters). [Remark]

Honestly the fact that this worked at all is incredible ๐Ÿ˜ญ Over 25 years of computing hardware and FITS evolution, that ability to transfer the data content hasn't degraded one bit ๐ŸŽ‰

The structure of FITS files is super general, and that really fits astronomical data ๐ŸŒˆ FITS stores observations across the entire electromagnetic spectrum, from radio to gamma rays, from both ground-based observatories and spacecraft ๐Ÿ”ญ It also communicates physical properties other than radiation intensity. Those might be inferred from observations or calculated by theoretical models. The pedigree of data in a FITS file usually depends on the discipline. FITS can hold raw and uncalibrated data, fully reduced and calibrated data, or both ๐Ÿง Still, the FITS standard guarantees that the syntactic content of the data and metadata stays unambiguously available to posterity โณ

Observatories have built lots of local conventions for storing and transferring data specific to their own instruments and purview. Application software for handling FITS files from different parts of the electromagnetic spectrum has mostly been disjoint ๐Ÿ˜ตโ€๐Ÿ’ซ For a FITS file with multiple HDUs, there are no widely established conventions for what the data sets mean, how they relate to each other, or how they're suggested to be used. Recognizing any local conventions used in FITS data has often relied on heuristics from the additional (non-standard) keyword/value pairs ๐Ÿ•ต๏ธโ€โ™€๏ธ Fully understanding the semantic content of a FITS file usually requires an external data dictionary ๐Ÿ“–๐Ÿ’€

4.5. Application Programming Interfaces to FITS โœจ

okay so the FITS definition is all about the exact bits inside the files ๐Ÿง , but bestie, there are SO many APIs that make working with FITS files way easier ๐Ÿ’…. Interfaces exist for lots of languages and operating systems, and here's a partial list ๐ŸŽ€:

CFITSIO      http://heasarc.gsfc.nasa.gov/fitsio/
fitsTcl      http://heasarc.gsfc.nasa.gov/ftools/fv/fitsTcl_home.html
WCSLIB       http://www.atnf.csiro.au/people/mcalabre/WCS/
PyFITS       http://www.stsci.edu/resources/software_hardware/pyfits
WCSTools     http://tdc-www.harvard.edu/software/wcstools/
FUNTOOLS     http://hea-www.harvard.edu/RD/funtools/
IDLASTRO     http://idlastro.gsfc.nasa.gov/
fitsy        http://cfa-www.harvard.edu/~john/fitsy/
IUEDAC       http://archive.stsci.edu/iue/iuedacfits.html
Mathematica  http://documents.wolfram.com/v5/Built-inFunctions/
                    GraphicsAndSound/ImportAndExport/
                    AdditionalInformation/Import.html
MatLab       http://www.mathworks.com/access/helpdesk/help/techdoc/
                    ref/fitsread.shtml?cmdname=fitsread

for more APIs, go check the current lists at ๐ŸŒธ

http://fits.gsfc.nasa.gov/fits_libraries.html

and if you want to see which applications actually use FITS, the IANA registrations of the media types have a list of them ๐Ÿฆ‹ iykyk.

4.6. FITS File Conformance Testing ๐Ÿ”ฅ

so if you want to check whether your FITS files follow the Definition of FITS rules [NOST, WCS1, WCS2], there's an app called "fitsverify" ๐Ÿ’–. You can grab it from here:

http://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/

it comes as executable binary files for Solaris, Linux, and Windows, and there's source code too ๐Ÿซถ. ngl though, "fitsverify" has NOT been endorsed by the IAUFWG ๐Ÿ’€, so just keep that in mind. Also, the designer of the program was the Secretary of the Technical Panel that produced the published FITS standard [NOST], so she literally knows the spec inside out ๐Ÿ‘‘.

4.7. Archives That Distribute FITS Files ๐ŸŒˆ

as I mentioned back in section 4.3 of this RFC, those massive (multi-terabyte) data archives that hold and/or distribute FITS files are a big reason the FITS standard stays so stable ๐Ÿงš. There are TONS of publicly available archives of FITS files, from both space and ground-based observations, and they cover basically the whole electromagnetic spectrum, from radio all the way up to gamma-ray wavelengths โญ. Here are some examples, listed in no particular order ๐Ÿ“:

Telescope(s)   URLs for archive access
------------   ------------------------------------------------------
KPNO,CTIO,..   http://archive.noao.edu/nsa/
VLT,HST,..     http://archive.eso.org/
Subaru         http://smoka.nao.ac.jp/
SDSS           http://www.sdss.org/dr3/
CFHT           http://cadcwww.dao.nrc.ca/cfht/cfht.html
VLA,VLBA,GBT   http://e2e.aoc.nrao.edu/archive/archive_describe.html
HST,MAST       http://archive.stsci.edu/
HEASARC        http://heasarc.gsfc.nasa.gov/w3browse/
Chandra        http://cxc.harvard.edu/cda/
LaPalma        http://archive.ast.cam.ac.uk/ingarch/
BIMA           http://bimaarch.ncsa.uiuc.edu/
Keck-DEIMOS    http://archive.deep.ucolick.org/
ComptonGRO     http://cossc.gsfc.nasa.gov/archive/index.html
Spitzer,..     http://www.ipac.caltech.edu/
AAT            http://www.aao.gov.au/archive/
HIPASS         http://www.atnf.csiro.au/research/multibeam/
                       multibeam.html
JCMT           http://salish.dao.nrc.ca:8080/jcmt/intro.html
COBE,WMAP      http://lambda.gsfc.nasa.gov/
EVN            http://www.jive.nl/archive/scripts/listarch.php
Gemini         http://gemini.ast.cam.ac.uk/sciops/data/dataIndex.html
XMM-Newton     http://xmm.vilspa.esa.es/external/xmm_data_acc/xsa/

5. IANA Considerations ๐Ÿ’œ

okay so the full FITS standard is pretty general, which means it needs the media type "application/fits" ๐ŸŽ€. BUT, bestie, a whole lot of FITS files are really just about carrying one data array in the PHDU ๐Ÿฆ‹, and those arrays are super often 2-dimensional images ๐ŸŒธ. Several common image viewers already show single-HDU FITS files, and the prototypes for virtual observatory projects say that data from web services should be carried in the data array in the PHDU โœจ. Those uses are why we're registering a second media type, "image/fits", for files that use the subset of the standard from the original FITS standard paper [FITS] ๐Ÿ’–.

We have to note that the media type "image/gif" [MIME2] allows raster images that are three dimensional, since animated GIFs have two spatial dimensions and one temporal dimension ๐ŸŒˆ. We also note that "image/vnd.dwg" and "image/vnd.dxf" can hold data with three-dimensional vectors and curves, plus objects made with constructive solid geometry ๐ŸงŠ. Following those precedents for the "image" media type ๐Ÿ‘‘, we specify that "image/fits" MAY be used to describe FITS PHDUs that have other than two dimensions ๐Ÿซถ. Still, we expect that most files described as "image/fits" will have two-dimensional (NAXIS=2) PHDUs ๐Ÿ’….

5.1. Registration of application/fits โœจ

okay bestie, this is the actual registration form for the media type ๐ŸŽ€ it tells IANA everything about application/fits, so let's go through it.

  • To: ietf-types@iana.org
  • Subject: Registration of Standard MIME Media type application/fits
  • MIME media type name: application
  • MIME subtype name: fits
  • Required parameters: none
  • Optional parameters: none
  • Encoding considerations: binary

FITS files can be quite large ๐Ÿ˜ so when they get sent over HTTP, it may be efficient to use content-coding or transfer-coding values like gzip, compress, or deflate. just saying ๐Ÿ’…

Security considerations:

okay this part is serious but also kinda iconic ๐Ÿ’€ FITS is a way to carry arrays and tables of data, plus keyword/value pairs of metadata. the standard FITS keywords are either mandatory or reserved. mandatory keywords give the info you need to read the data correctly, while reserved keywords are just standard bits of metadata ๐Ÿงš so the current standard FITS keywords do not pose security risks.

BUT ๐Ÿšจ a FITS file author can throw in extra keywords whose meaning the standard doesn't describe. people who swap FITS files can also use their own local conventions, where keywords and their values make the receiver do something. some existing local conventions use keywords to ask for other files and/or URIs to be read. other local conventions use keywords to change the state of a telescope and/or instrument ๐Ÿ˜ญ the parties using these conventions SHOULD analyze what security implications they have. this is non-negotiable, bestie ๐Ÿ”ฅ

Interoperability considerations:

FITS files have been moving between wildly different computers since 1979 ๐ŸŒˆ the most likely problem a FITS application runs into is not having enough computational resources to handle a very large FITS file. big files, big feelings ๐Ÿฅฒ

Published specification:

the spec was published as a series of papers in refereed astronomical journals, so it's very academic and very slay ๐Ÿ“š

Hanisch, R., et al., "Definition of the Flexible Image Transport System (FITS)", Astronomy & Astrophysics, 376, p. 359, 2001.

Greisen, E. and M. Calabretta, "Representations of world coordinates in FITS", Astronomy & Astrophysics, 395, p. 1061, 2002.

Calabretta, M. and E. Greisen, "Representations of celestial coordinates in FITS", Astronomy & Astrophysics, 395, p. 1077, 2002.

you can also find copies of these specs at these places:

http://fits.gsfc.nasa.gov/
http://archive.stsci.edu/fits/
http://www.cv.nrao.edu/fits/
http://heasarc.gsfc.nasa.gov/docs/heasarc/fits.html

Applications that use this media type:

there are tons of astronomical image viewing and data reduction apps, and this list is not the whole thing ๐Ÿ”ญ here are some of them:

IRAF     http://iraf.noao.edu/
AIPS     http://www.aoc.nrao.edu/aips/
AIPS++   http://aips2.nrao.edu/
MIDAS    http://www.eso.org/projects/esomidas/
ds9      http://hea-www.harvard.edu/RD/ds9/
fv       http://heasarc.gsfc.nasa.gov/ftools/fv/
Aladin   http://aladin.u-strasbg.fr/
Starlink http://star-www.rl.ac.uk/
Miriad   http://bima.astro.umd.edu/miriad/
STSDAS   http://www.stsci.edu/resources/software_hardware/stsdas
PROS     http://hea-www.harvard.edu/PROS/pros.html
CIAO     http://cxc.harvard.edu/ciao/
XANADU   http://heasarc.gsfc.nasa.gov/docs/xanadu/xanadu.html
HESSI    http://hesperia.gsfc.nasa.gov/ssw/hessi/doc/
FITSview http://www.nrao.edu/software/fitsview/
XMM-SAS  http://xmm.vilspa.esa.es/external/xmm_sw_cal/sas_frame.shtml
the tea: a lot of these apps aren't built to act as viewers of "application/fits" files inside web browsers yet ๐Ÿซ 

Additional information:

a FITS file described with the media type application/fits SHOULD conform to the published standards for FITS files, as decided by convention and agreement inside the international FITS community. nothing else is required for what goes inside a file labeled application/fits ๐Ÿ’–

a FITS file labeled application/fits can have any number of conforming extension header and data units (XHDUs) after its mandatory primary header and data unit (PHDU) ๐Ÿง the XHDUs can be one of the standard types (IMAGE, TABLE, and BINTABLE) or any other type that meets the "Requirements for Conforming Extensions" (section 4.4.1 of [NOST]). the PHDU or any IMAGE XHDU can have zero to 999 dimensions, with zero or more pixels along each dimension ๐Ÿ“

the PHDU can use the random groups convention, where the first axis has a dimension of zero and the keywords GROUPS, PCOUNT, and GCOUNT show up in the header. NAXIS1=0 and GROUPS=T is the signature of random groups. see section 7 of the Definition of FITS paper [NOST] for more ๐Ÿฆ‹

Recommendations for application writers:

okay app devs, listen up ๐Ÿ‘‘ an application meant to handle application/fits SHOULD be able to show a user a manifest of every HDU in the file, along with all of the keyword/value pairs from each HDU.

an application meant to handle application/fits SHOULD be prepared to run into XHDUs that hold either ASCII or binary tables, and to give users access to their elements ๐Ÿ“Š

an application that can change FITS files or fetch them from an external service SHOULD be capable of writing those files to local storage ๐Ÿ’พ

figuring out what the data in each HDU means and how it's meant to be used usually takes heuristics that try to work out which local conventions the file author used ๐Ÿ•ต๏ธโ€โ™€๏ธ it's giving detective energy ngl

as examples, files with media type application/fits might hold any of these:

  • An empty PHDU (containing zero data elements) followed by a table HDU that contains a catalog of celestial objects.
  • An empty PHDU followed by a table HDU that encodes a series of time-tagged photon events from an exposure using an X-ray detector.
  • An empty PHDU followed by a series of IMAGE HDUs containing data from an exposure taken by a mosaic of CCD detectors.
  • An empty PHDU followed by a series of table HDUs that contain a snapshot of the state of a relational database.
  • A PHDU containing a single image along with keyword/value pairs of metadata.
  • A PHDU with NAXIS1=0 and GROUPS=T followed by random groups data records of complex fringe visibilities.

Magic number(s): SIMPLE = T

Jeff Uphoff of the National Radio Astronomy Observatory (NRAO) added database entries to the magic number file that the Unix "file" command uses ๐ŸŽ‰ magic number files with those entries ship with a bunch of Unix-like operating systems. on top of recognizing a FITS file by the string above, the Uphoff entries also recognize the data type of the pixels in the PHDU. he's a legend fr fr ๐Ÿซถ

File extension(s): fits

this file extension SHOULD NOT be interpreted as a prescription ๐Ÿ™ˆ the FITS standard came from the era of magnetic tape, so it doesn't set any naming rules for files on disk. lots of FITS sites have long-running habits where files are assumed to be FITS based on context, and at those sites file extensions usually show what the content is rather than the data format ๐Ÿ“

if there's no other info, it's pretty safe to assume that a file name ending in .fits is meant to be a FITS file. there are other popular extensions too, like .fit, .fts, and many more that aren't worth listing in a media type registration ๐ŸŒธ

Intended usage: Common

Persons to contact for further information:

  • "Steve Allen" <sla@ucolick.org>
  • "Don Wells" <dwells@nrao.edu>

Author/Change controller:

  • "Steve Allen" <sla@ucolick.org>

the IAU FITS Working Group may authorize changes to this document ๐Ÿ‘‘โœจ


that's the registration form, bestie ๐ŸŒˆ next up, the rest of the document ๐Ÿ’œ

5.2. Registration of image/fits ๐ŸŽ€

Okay bestie, this is the registration for image/fits, the media type that basically says "hey, this is a FITS file!" ๐Ÿ’– Here's the template, copied exactly:

To: ietf-types@iana.org
Subject: Registration of Standard MIME Media type image/fits

MIME media type name: image

MIME subtype name: fits

Required parameters: none

Optional parameters: none

Encoding considerations: binary

FITS files can be quite large ๐Ÿ˜ญ. When they get sent over HTTP, it may be efficient for the transaction to use content-coding or transfer-coding values like "gzip", "compress", or "deflate" ๐Ÿ—œ๏ธโœจ.

Security considerations:

FITS is a way to transport arrays and tables of data, plus keyword/value pairs of metadata ๐Ÿ“ฆ. The standard FITS keywords are either mandatory or reserved. Mandatory keywords give you the info you need to read the data correctly, and reserved keywords are just standard bits of metadata ๐Ÿงฉ. So the current standard FITS keywords don't pose security risks, which is honestly so reassuring ๐Ÿฅน.

Now the plot twist ๐Ÿ‘€. A FITS file author can add extra keywords with meanings the standard doesn't describe. Parties who swap FITS files can also use local conventions where keywords and their values trigger actions on the recipient's side. Some existing local conventions use these keywords to ask for other files or URIs to be read. Others use them to change the state of a telescope or instrument ๐Ÿ”ญ. The security implications of local conventions like these SHOULD be analyzed by the parties using them.

Quick heads-up, bestie: those local conventions are where the real risk lives. Analyze them before you trust them ๐Ÿšจ.

Interoperability considerations:

FITS files have been moving between wildly different computers since 1979 ๐Ÿ–ฅ๏ธ๐Ÿ’•. The difficulty a FITS application is most likely to run into is not having enough computing resources to handle a very large FITS file ๐Ÿ’€.

Published specification:

The specification for this content type is published as a series of papers in refereed astronomical journals ๐Ÿ“š. The references are:

Hanisch, R., et al., "Definition of the Flexible Image Transport
System (FITS)", Astronomy & Astrophysics, 376, p. 359, 2001.

Greisen, E.  and M.  Calabretta, "Representations of world
coordinates in FITS", Astronomy & Astrophysics, 395, p.  1061, 2002.

Calabretta, M. and E. Greisen, "Representations of celestial
coordinates in FITS", Astronomy & Astrophysics, 395, p. 1077, 2002.

You can find copies of these specs at these spots ๐ŸŒธ:

http://fits.gsfc.nasa.gov/
http://archive.stsci.edu/fits/
http://www.cv.nrao.edu/fits/
http://heasarc.gsfc.nasa.gov/docs/heasarc/fits.html

Applications that use this media type:

There are so many astronomical image viewing and data reduction apps ๐Ÿคฉ. Here are some of the big ones (the list isn't exhaustive, so there are more out there ๐ŸŒŒ):

IRAF     http://iraf.noao.edu/
AIPS     http://www.aoc.nrao.edu/aips/
AIPS++   http://aips2.nrao.edu/
MIDAS    http://www.eso.org/projects/esomidas/
ds9      http://hea-www.harvard.edu/RD/ds9/
fv       http://heasarc.gsfc.nasa.gov/ftools/fv/
Aladin   http://aladin.u-strasbg.fr/
Starlink http://star-www.rl.ac.uk/
Miriad   http://bima.astro.umd.edu/miriad/
STSDAS   http://www.stsci.edu/resources/software_hardware/stsdas
PROS     http://hea-www.harvard.edu/PROS/pros.html
CIAO     http://cxc.harvard.edu/ciao/
XANADU   http://heasarc.gsfc.nasa.gov/docs/xanadu/xanadu.html
HESSI    http://hesperia.gsfc.nasa.gov/ssw/hessi/doc/
FITSview http://www.nrao.edu/software/fitsview/
XMM-SAS  http://xmm.vilspa.esa.es/external/xmm_sw_cal/sas_frame.shtml

Non-astronomical FITS image display apps include ๐ŸŽจ:

netpbm          http://netpbm.sourceforge.net/
gimp            http://www.gimp.org/
IDL             http://www.rsinc.com/
ImageMagick     http://www.imagemagick.com/
Mathematica     http://www.wolfram.com/
MatLab          http://www.mathworks.com/
xv              http://www.trilon.com/xv/xv.html

There are also two FITS plug-ins for Adobe Photoshop ๐Ÿ“ธ, which you can get at http://astroshed.com/fitsplug/fitsplug.htm and http://www.spacetelescope.org/projects/fits_liberator/. The Photoshop site itself is http://www.adobe.com/products/photoshop/.

At the moment, a lot of the apps above weren't built to work as "image/fits" viewers alongside web browsers ๐ŸŒ๐Ÿ’ญ.

Additional information:

A FITS file described as "image/fits" SHOULD have a PHDU with positive integer values for the NAXIS and NAXISn keywords, so it SHOULD contain at least one pixel ๐Ÿ–ผ๏ธ. Files with 4 or more non-degenerate axes (NAXISn>1) SHOULD be described as "application/fits", not "image/fits" ๐Ÿ“. (In rare cases it may be appropriate to describe a NULL image, which is a dataless container for FITS keywords with NAXIS=0 or NAXISn=0, or an image with 4+ non-degenerate axes, as "image/fits". This usage is discouraged because those files may confuse simple image viewer apps ๐Ÿ™ˆ.)

FITS files declared as "image/fits" MAY also have one or more conforming XHDUs after their PHDUs ๐Ÿง. These extension HDUs MAY contain standard, non-linear, world coordinate system (WCS) info in the form of tables or images ๐Ÿ—บ๏ธ. The extension HDUs MAY also hold other, non-standard metadata about the image in the PHDU, in the form of keywords and tables.

A FITS file described as "image/fits" SHOULD be principally meant to communicate the single data array in the PHDU ๐ŸŽฏ. That means "image/fits" SHOULD NOT be applied to FITS files containing MEF (multi-exposure-frame) mosaic images ๐Ÿšซ. Also, random groups files MUST be described as "application/fits" and not as "image/fits" ๐Ÿ™…โ€โ™€๏ธ.

A FITS file described as "image/fits" is also valid as a file of media type "application/fits" ๐Ÿ’œ. Which one you pick depends on the context and what you intend to do with it.

Recommendations for application writers:

An app that is meant to handle "image/fits" SHOULD be able to show a user a manifest of all the HDUs in the file, along with all the keyword/value pairs from each HDU ๐Ÿ“‹. An app writer MAY choose to ignore HDUs past the PHDU, but even then the app SHOULD be able to show the user the keyword/value pairs from the PHDU ๐Ÿซถ.

Note that an app meant to render "image/fits" for a user to look at has way more responsibility than an app that handles something like "image/tiff" or "image/gif" ๐Ÿ˜ค. FITS data arrays hold elements that usually represent the values of a physical quantity at some coordinate location ๐ŸŒก๏ธ. So they don't need any pixel rendering info like transfer functions, and there's no mechanism for color look-up tables. An app SHOULD provide this functionality, either automatically with a more or less fancy algorithm, or interactively, letting the user pick from various degrees of choice ๐ŸŽ›๏ธ.

Also, the elements in a FITS data array may be integers or floating-point numbers ๐Ÿ”ข. The dynamic range of the array values may go beyond what the display medium and the eye can handle, and their distribution may be super nonuniform ๐ŸŒˆ. Logarithmic, square-root, and quadratic transfer functions, plus histogram equalization techniques, have proven helpful for rendering FITS data arrays โœจ. Some array elements may have values that show their data is undefined or invalid, and those should be rendered distinctly ๐Ÿ‘€. Via WCS Paper I [WCS1], the standard lets you use CTYPEnnn = 'COMPLEX' to say a data array holds complex numbers (future revisions might allow other elements, like quaternions or general tensors ๐Ÿง ).

Three-dimensional data arrays (NAXIS=3 with NAXIS1, NAXIS2 and NAXIS3 > 1) are the main characters of this section ๐Ÿ‘‘. Apps meant to handle "image/fits" MAY default to showing the first 2D plane of the image cube, or they MAY default to presenting it like an animated GIF ๐ŸŽž๏ธ, or they MAY show the data cube as a mosaic of "thumbnail" images ๐Ÿงฉ. Even without WCS indication of a temporal axis, the time-lapse movie-looping display technique can work really well, and app writers SHOULD consider offering it for all three-dimensional arrays ๐ŸŽฌ.

An "image/fits" PHDU with NAXIS=1 is describing a one-dimensional thing, like a spectrum or a time series ๐Ÿ“ˆ. Apps meant to handle "image/fits" MAY default to showing that image as a graphical plot, instead of a two-dimensional picture with a single row ๐Ÿ’….

An app that can't handle an image with any dimensionality other than 2 SHOULD gracefully tell its users about its limits when it hits NAXIS=1 or NAXIS=3 cases ๐Ÿ™, while still giving them access to the keyword/value pairs.

FITS files with degenerate axes (that is, one or more NAXISn=1) MAY be described as "image/fits". But the first axes SHOULD be non-degenerate, meaning the degenerate axes SHOULD be the highest dimensions ๐Ÿ“. An algorithm built to render only two-dimensional images can display a NAXIS=3 or NAXIS=4 FITS array where one or two of the axes are a single pixel, and an app writer SHOULD consider coding that capability in ๐Ÿ› ๏ธ. Writers of new apps that generate FITS files meant to be described as "image/fits" SHOULD consider using the WCSAXES keyword [WCS1] to declare the dimensionality of those degenerate axes ๐Ÿงƒ. That way NAXIS can be used to convey the number of non-degenerate axes ๐Ÿ’–.

Magic number(s):

SIMPLE  =                    T

Jeff Uphoff of the National Radio Astronomy Observatory (NRAO) contributed database entries for the magic number file the Unix "file" command uses ๐Ÿง. Magic number files with these entries ship with a bunch of Unix-like operating systems ๐Ÿ’ป. Besides recognizing a FITS file with the string above, the Uphoff entries also recognize the data type of the pixels in the PHDU ๐Ÿ”.

File extension(s): fits

This file extension SHOULD NOT be interpreted as a prescription ๐Ÿ™…โ€โ™€๏ธ.

The FITS standard started in the era of magnetic tape, when files were stored and swapped on tape ๐Ÿ“ผ. It doesn't prescribe any naming for files on disk. Lots of sites in the FITS community have long-standing practices where files are presumed to be FITS based on context ๐Ÿ—‚๏ธ. File extensions at those sites usually tell you what the content is, not what format it's in ๐Ÿค“.

Without other info, it's reasonably safe to assume a file name ending in ".fits" is meant to be a FITS file ๐Ÿฅฐ. There are other commonly used extensions too, like ".fit", ".fts", and many more that don't belong in a media type registration ๐ŸŒŸ.

Intended usage: Common

Persons to contact for further information:

   "Steve Allen"     <sla@ucolick.org>
   "Don Wells"       <dwells@nrao.edu>

Author/Change controller:

   "Steve Allen"     <sla@ucolick.org>

The IAU FITS Working Group may authorize changes to this document ๐ŸŒˆ.


6. References โœจ

6.1. Normative References ๐Ÿ“šโœจ

Okay bestie, these are the sources the spec needs you to have read, so no skipping! ๐Ÿค“ Every entry is listed with its full details:

  • [Require] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [MIME1] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
  • [MIME2] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996.
  • [MIME4] Freed, N., Klensin, J., and J. Postel, "Multipurpose Internet Mail Extensions (MIME) Part Four: Registration Procedures", BCP 13, RFC 2048, November 1996.
  • [HTTP] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
  • [NOST] Hanisch, R., et al., "Definition of the Flexible Image Transport System (FITS)", Astronomy & Astrophysics, 376, p. 359, 2001.
  • [WCS1] Greisen, E. and M. Calabretta, "Representations of world coordinates in FITS", Astronomy & Astrophysics, 395, p. 1061, 2002.
  • [WCS2] Calabretta, M. and E. Greisen, "Representations of celestial coordinates in FITS", Astronomy & Astrophysics, 395, p. 1077, 2002.

6.2. Informative References ๐ŸŒธ

These ones are the lore! They're not required reading, but they're the origin story of FITS and seriously worth the rabbit hole. ๐Ÿฆ‹

  • [FITS] Wells, D., et al., "FITS: A Flexible Image Transport System", Astronomy & Astrophysics Supplement, 44, p. 363, 1981.
  • [GROUPS] Greisen, E. and R. Harten, "An extension of FITS for groups of small arrays of data", Astronomy & Astrophysics Supplement, 44, p. 371, 1981.
  • [XTENSION] Grosbol, P., et al., "Generalized extensions and blocking factors for FITS", Astronomy & Astrophysics Supplement, 73, p. 359, 1988.
  • [IAUFWG] McNally, D., ed., "Transactions of the IAU, Vol. XXB 1988, Proceedings of the Twentieth General Assembly Baltimore 1988", Kluwer Academic, p. 52 (Resolution B2), 1988.
  • [TABLE] Harten, R., et al., "The FITS tables extension", Astronomy & Astrophysics Supplement, 73, p. 365, 1988.
  • [IMAGE] Ponz, J., et al., "The FITS image extension", Astronomy & Astrophysics Supplement, 105, p. 53, 1994.
  • [BINTABLE] Cotton, W., et al., "Binary table extension to FITS", Astronomy & Astrophysics Supplement, 113, p. 159, 1995.
  • [Remark] Greisen, E., "FITS: A remarkable achievement in information exchange" in "Information Handling in Astronomy -- Historical Vistas", A. Heck, ed., Kluwer Academic, p. 71, 2003.
  • [IVOA] The International Virtual Observatory Alliance, http://www.ivoa.net/
  • [NVO] The US National Virtual Observatory, http://www.us-vo.org/
  • [AstroGrid] The UK AstroGrid, http://www.astrogrid.org/
  • [AVO] The European Astrophysical Virtual Observatory, http://www.euro-vo.org/
  • [ASU] Albrecht, M., et al., "Astronomical Server URL", http://vizier.u-strasbg.fr/doc/asu.html, 1996.
  • [SIAP] Tody, D., et al., "Simple Image Access Prototype Specification", http://www.us-vo.org/pubs/, 2002.
  • [TIFF] Adobe Systems Incorporated, "TIFF Revision 6.0", http://partners.adobe.com/asn/developer/pdfs/tn/TIFF6.pdf 1992.
  • [GeoTIFF] Ritter, N. and M. Ruth, "GeoTIFF Format Specification", http://www.remotesensing.org/geotiff/geotiff.html, 2000.

7. Security Considerations ๐Ÿ”๐Ÿ’€

Okay, this one is short but it is SERIOUS. Security considerations are not optional vibes! ๐Ÿ˜ญ

The security considerations of interchanging FITS files are discussed above within the text of the IANA registration for each media type. So if you want the full security tea โ˜•, go back up to the registrations in Section 5.1 and Section 5.2. ๐Ÿซถ

8. Contributors ๐Ÿงš

Several individuals have made significant contributions to the content and clarity of this text:

  • Francois Ochsenbein (Observatoire Astronomique de Strasbourg) ๐Ÿ’œ
  • Clive Davenhall (Institute for Astronomy of the Royal Observatory Edinburgh) ๐Ÿ’œ
  • Tom McGlynn (Laboratory for High Energy Astrophysics of the NASA Goddard Space Flight Center) ๐Ÿ’œ
  • Lucio Chiappetti (Milan section of the Italian Istituto di Astrofisica Spaziale e Fisica Cosmica) ๐Ÿ’œ
  • William Pence (NASA High Energy Astrophysics Science Archive Research Center) ๐Ÿ’œ
  • Arnold Rots (High Energy Astrophysics Division of the Smithsonian Astrophysical Observatory) ๐Ÿ’œ
  • Doug Tody (National Radio Astronomy Observatory) ๐Ÿ’œ
  • Bob Hanisch (Space Telescope Science Institute) ๐Ÿ’œ
  • Mark Calabretta (Australia Telescope National Facility) ๐Ÿ’œ

9. Acknowledgements ๐Ÿซถ๐ŸŒˆ

This document originated when William Joye of the Research and Development Group at the Smithsonian Astrophysical Observatory High Energy Astrophysics Division discovered many experimental and unofficial MIME media types being used by various agencies. Iconic origin story, honestly. ๐Ÿ‘‘

Jeff Uphoff of the National Radio Astronomy Observatory (NRAO) contributed the FITS entries for the magic number file that permits the Unix-like "file" command on many systems to identify a FITS file. ๐Ÿ”โœจ

Nelson Zarate verified that the fgread and fgwrite programs are able to store hierarchical directories containing files with arbitrary MIME media types within a HDU of a FITS file. The fgread and fgwrite programs are part of the FITSUTIL IRAF external package (version dated September 1999) written by N. Zarate, D. Tody, and R. Seaman at National Optical Astronomy Observatory (NOAO). Main character energy for these folks. ๐ŸŽ€

Authors' Addresses ๐Ÿ“ฎ

   Steven L. Allen
   UCO/Lick Observatory
   University of California
   Santa Cruz, CA  95064  USA

   Phone: +1 831 459 3046
   EMail: sla@ucolick.org


   Donald C. Wells
   National Radio Astronomy Observatory
   520 Edgemont Road
   Charlottesville, Virginia  22903-2475  USA

   Phone: +1 434 296 0277
   EMail: dwells@nrao.edu

Copyright (C) The Internet Society (2005).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property ๐Ÿงพ

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement ๐Ÿ’–

Funding for the RFC Editor function is currently provided by the Internet Society. โœจ๐Ÿฆ‹

๐Ÿ“ฆ where this page comes from