NCKS(1)

📦
version 5.3.9 · 8 versions
, in ubuntu stonking

👑 NCKS — netCDF Kitchen Sink 💅

okay bestie, so this is the NCKS(1) page from the General Commands Manual, and it is a lot of page, but don't panic, we're gonna get through it together 🫶

✨ NAME ✨

ncks — netCDF Kitchen Sink 🍳

📜 SYNTAX 📜

ncks [-3] [-4] [-5] [-6] [-7] [-A] [-a] [--area_wgt] [-b bnr_fl] [--bfr
sz_byt] [-C] [-c] [--cal] [--cdl] [--chk_bnd] [--chk_chr] [--chk_map]
[--chk_mss] [--chk_nan] [--chk_xtn] [--cmp cmp_sng] [--cnk_byt sz_byt]
[--cnk_csh sz_byt] [--cnk_dmn nm,sz_lmn] [--cnk_map map] [--cnk_min
sz_byt] [--cnk_plc plc] [--cnk_scl sz_lmn] [-D dbg_lvl] [-d dim,[
min][,[ max]][,[ stride]]] [-F] [--fl_fmt=fmt] [--fix_rec_dmn dim]
[--fmt_val fmt] [--fpe] [-G gpe_dsc] [-g grp[,...]]  [--gaa att_name=
att_val]] [--gad att1[,...]]  [--grp_xtr_var_xcl] [-H] [-h] [--hdn]
[--hdr_pad sz_byt] [--hpss_try] [--hrz_s1d hrz.nc] [--is_hrz var]
[--json] [--jsn_fmt lvl] [-k] [-l path] [-M] [-m] [--map map-file]
[--md5] [--mk_rec_dmn dim] [--msa] [--no_blank] [--no_cll_msr]
[--no_frm_trm] [--no_tmp_fl] [-O] [-o output-file] [-P] [-p path]
[--prn_fl print-file] [-Q] [-q] [--qnt var1[, var2[,...]]= prc]]
[--qnt_alg alg_nm] [-R] [-r] [--rad] [--ram_all] [--rgr key= val] [--rnr
wgt] [-s format] [--s1d] [-t thr_nbr] [-u] [--uio] [--unn] [-V] [-v
var[,...]]  [--vrt_in vrt.nc] [--vrt_out vrt.nc] [-X box] [-x] [--xml]
input-file [ output-file]

like. look at her. that's not a synopsis that's a whole personality 😭

💖 DESCRIPTION 💖

so ncks combines every feature we could think of, except the kitchen sink, into one versatile utility to manipulate netCDF files 💅 it extracts a subset of the data from input-file and then either prints it as ASCII text to stdout, or writes (or pastes) it to output-file, or — main character energy — both.

ncks will print netCDF data in ASCII format to stdout, like ncdump, but bestie there are differences:

  • ncks prints data in a tabular format intended to be easy to search for the data you want
  • one datum per screen line 📏
  • with all dimension subscripts and coordinate values (if any) preceding the datum

and option -s lets you format the data using C-style format strings ✨

options -a, -F, -H, -M, -m, -Q, -q, -s, -u, and -V control the formatted appearance of the data 🎀

ncks will extract (and optionally create a new netCDF file comprised of) only selected variable from the input file, like ncextr but with these differences: only variables and coordinates may be specifically included or excluded — all global attributes and any attribute associated with an extracted variable will be copied to the screen and/or output netCDF file 💖 options -c, -C, -v, and -x control which variables are extracted.

ncks will extract hyperslabs from the specified variables 🔥 in fact ncks implements the nccut specification exactly. option -d controls the hyperslab specification.

input dimensions that are not associated with any output variable will not appear in the output netCDF 👑 this feature removes superfluous dimensions from a netCDF file. slay.

ncks will append variables and attributes from the input-file to output-file if output-file is a pre-existing netCDF file whose relevant dimensions conform to dimension sizes of input-file 🦋 the append features of ncks are intended to provide a rudimentary means of adding data from one netCDF file to another, conforming, netCDF file. when naming conflicts exist between the two files, data in output-file is usually overwritten by the corresponding data from input-file 😭

so like, please back up your output-file bestie, because valuable data can get accidentally overwritten and we do NOT want that for you 💀

if output-file exists, the user will be queried whether to overwrite, append, or exit the ncks call completely 🙈 choosing overwrite destroys the existing output-file and creates an entirely new one from the output of the ncks call.

append has differing effects depending on the uniqueness of the variables and attributes output by ncks:

  • if a variable or attribute extracted from input-file does not have a name conflict with the members of output-file, then it will be added to output-file without overwriting any of the existing contents of output-file 🫶 in this case the relevant dimensions must agree (conform) between the two files; new dimensions are created in output-file as required.
  • when a name conflict occurs, a global attribute from input-file will overwrite the corresponding global attribute from output-file 🌸
  • if the name conflict occurs for a non-record variable, then the dimensions and type of the variable (and of its coordinate dimensions, if any) must agree (conform) in both files ✅ then the variable values (and any coordinate dimension values) from input-file will overwrite the corresponding variable values (and any coordinate dimension values, if any) in output-file.

since there can only be one record dimension in a file, the record dimension must have the same name (but not necessarily the same size) in both files if a record dimension variable is to be appended 🔥 if the record dimensions are of differing sizes, the record dimension of output-file will become the greater of the two record dimension sizes, the record variable from input-file will overwrite any counterpart in output-file, and fill values will be written to any gaps left in the rest of the record variables (I think) 💀

in all cases variable attributes in output-file are superseded by attributes of the same name from input-file, and left alone if there is no name conflict ⭐

some users may wish to avoid interactive ncks queries about whether to overwrite existing data 🙈 for example, batch scripts will fail if ncks does not receive responses to its queries. options -O and -A are available to force overwriting existing files and variables, respectively 👑

🍓 Options specific to ncks 🍓

the following list provides a short summary of the features unique to ncks 💅

  • -a — do not alphabetize extracted fields. by default, the specified output variables are extracted, printed, and written to disk in alphabetical order 🔤 this tends to make long output lists easier to search for particular variables. specifying -a results in the variables being extracted, printed, and written to disk in the order in which they were saved in the input file 🦋 so -a retains the original ordering of the variables. periodt.
  • -d — dim,[ min][,[ max]][,[ stride]] — add stride argument to hyperslabber 💅
  • -H — print data to screen 📺 the default behavior is to print data to screen if no netCDF output file is specified. use -H to print data to screen if a netCDF output is specified (the same behavior applies to -m). unless otherwise specified (with -s), each element of the data hyperslab is printed on a separate line containing the names, indices, and, values, if any, of all of the variables dimensions 📏

the dimension and variable indices refer to the location of the corresponding data element with respect to the variable as stored on disk (i.e., not the hyperslab).

so like, here's the vibe:

% ncks -H -C -v three_dmn_var in.nc
lat[0]=-90 lev[0]=100 lon[0]=0 three_dmn_var[0]=0
lat[0]=-90 lev[0]=100 lon[1]=90 three_dmn_var[1]=1
lat[0]=-90 lev[0]=100 lon[2]=180 three_dmn_var[2]=2
 ...
lat[1]=90 lev[2]=1000 lon[1]=90 three_dmn_var[21]=21
lat[1]=90 lev[2]=1000 lon[2]=180 three_dmn_var[22]=22
lat[1]=90 lev[2]=1000 lon[3]=270 three_dmn_var[23]=23

printing the same variable with the -F option shows the same variable indexed with Fortran conventions ✨

% ncks -F -H -C -v three_dmn_var in.nc
lon(1)=0 lev(1)=100 lat(1)=-90 three_dmn_var(1)=0
lon(2)=90 lev(1)=100 lat(1)=-90 three_dmn_var(2)=1
lon(3)=180 lev(1)=100 lat(1)=-90 three_dmn_var(3)=2
 ...

printing a hyperslab does not affect the variable or dimension indices, since these indices are relative to the full variable (as stored in the input file), and the input file has not changed 👀 however, if the hyperslab is saved to an output file and those values are printed, the indices will change:

% ncks -H -d lat,90.0 -d lev,1000.0 -v three_dmn_var in.nc out.nc
lat[1]=90 lev[2]=1000 lon[0]=0 three_dmn_var[20]=20
lat[1]=90 lev[2]=1000 lon[1]=90 three_dmn_var[21]=21
lat[1]=90 lev[2]=1000 lon[2]=180 three_dmn_var[22]=22
lat[1]=90 lev[2]=1000 lon[3]=270 three_dmn_var[23]=23
% ncks -H out.nc
lat[0]=90 lev[0]=1000 lon[0]=0 three_dmn_var[0]=20
lat[0]=90 lev[0]=1000 lon[1]=90 three_dmn_var[1]=21
lat[0]=90 lev[0]=1000 lon[2]=180 three_dmn_var[2]=22
lat[0]=90 lev[0]=1000 lon[3]=270 three_dmn_var[3]=23
  • -M — print to screen the global metadata describing the file 🌈 this includes file summary information and global attributes.
  • -m — print variable metadata to screen (similar to ncdump -h) 👑 this displays all metadata pertaining to each variable, one variable at a time.
  • -Q — toggle printing of dimension indices and coordinate values when printing arrays 💅 the name of each variable will appear flush left in the output. this is useful when trying to locate specific variables when displaying many variables with different dimensions. the mnemonic for this option is "quiet" 🤫
  • -s — format — string format for text output. accepts C language escape sequences and printf() formats ✨
  • -u — accompany the printing of a variable's values with its units attribute, if it exists 🎀

💖 EXAMPLES 💖

okay bestie this is the part where it all clicks, fr fr ✨

🌸 Basic vibes

See literally everything in in.nc, but with that Fortran indexing energy:

ncks -H -F in.nc

🦋 Copying a whole file (glow-up edition)

Copy the netCDF file in.nc to file out.nc:

ncks -O in.nc out.nc

Now out.nc has all the data from in.nc, no cap. But bestie, there are two differences between them, and you should know:

  1. The history global attribute is gonna contain the command you used to make out.nc 👀
  2. The variables in out.nc will be defined in alphabetical order

Like, yes, the internal storage order of variables in a netCDF file should be invisible to you, the user. But lowkey there are situations where alphabetizing a file is actually useful (go read the -a switch description, iykyk 💅).

🌈 Printing variables, four iconic ways

Print variable three_dmn_var from in.nc with the default notations. Then print it as an un-annotated text column. Then print it signed with very high precision. Then print it as a comma-separated list. Watch this:

% ncks -H -C -v three_dmn_var in.nc
lat[0]=-90 lev[0]=100 lon[0]=0 three_dmn_var[0]=0
lat[0]=-90 lev[0]=100 lon[1]=90 three_dmn_var[1]=1
 ...
lat[1]=90 lev[2]=1000 lon[3]=270 three_dmn_var[23]=23
% ncks -s "%f\n" -H -C -v three_dmn_var in.nc
0.000000
1.000000
 ...
23.000000
% ncks -s "%+16.10f\n" -H -C -v three_dmn_var in.nc
   +0.0000000000
   +1.0000000000
 ...
  +23.0000000000
% ncks -s "%f, " -H -C -v three_dmn_var in.nc
0.000000, 1.000000,  ... , 23.000000,

The second and third options are chef's kiss when you're pasting data into text files like reports or papers 📝✨

🎀 Characters and strings (the chaotic part)

One dimensional arrays of characters stored as netCDF variables? They get automatically printed as strings, whether or not they're NUL-terminated. Like:

ncks -v fl_nm in.nc

And the %c formatting code is your bestie for printing multidimensional arrays of characters that represent fixed length strings:

ncks -H -s "%c" -v fl_nm_arr in.nc
🚨 bestie LISTEN. Using the %s format code on strings which are not NUL-terminated (and therefore are not technically strings at all) is likely to result in a core dump 💀 do not do this to yourself.

💜 Excluding stuff

Create netCDF out.nc containing all variables, and any associated coordinates, except variable time, from netCDF in.nc:

ncks -x -v time in.nc out.nc

Periodt. ✂️

👑 Extracting specific variables

Extract variables time and pressure from netCDF in.nc:

ncks -v time,pressure in.nc out.nc
ncks -C -v time,pressure in.nc out.nc

If out.nc doesn't exist yet, it'll just get created, no drama. If it does exist, you'll be prompted asking whether you wanna append to out.nc or overwrite out.nc 🙈

And here's the tea on those two versions:

  • The first one makes an out.nc containing time, pressure, and any coordinate variables associated with pressure
  • The second one (with -C) is guaranteed to contain only two variables: time and pressure. nothing else. zero extras. iconic.

🌸 Hyperslabs, my beloved

Create netCDF out.nc containing all variables from file in.nc, but restrict the dimensions of those variables to a hyperslab. And print (with -H) those hyperslabs to the screen for good measure, because we're nosy 💅

Here's the specified hyperslab, bestie, break it down with me:

  • the sixth value in dimension time
  • the half-open range lat <= 0.0 in coordinate lat
  • the half-open range lon >= 330.0 in coordinate lon
  • the closed interval 0.3 <= band <= 0.5 in coordinate band
  • and the cross-section closest to 1000.0 in coordinate lev

And the key thing, the thing you cannot forget: limits applied to coordinate values are specified with a decimal point, and limits applied to dimension indices do not have a decimal point. That's the whole trick, iykyk.

ncks -H -d time,5 -d lat,,0. -d lon,330., -d band,.3,.5 -d
lev,1000. in.nc out.nc

🧚 Wrapped coordinates (Greenwich meridian girlies rise up)

So imagine the domain of the monotonically increasing longitude coordinate lon is 0 < lon < 360. In this case, lon is an example of a wrapped coordinate 🌈

ncks will extract a hyperslab which crosses the Greenwich meridian simply by specifying the westernmost longitude as min and the easternmost longitude as max, like this:

ncks -d lon,260.,45. in.nc out.nc

obsessed with how easy that is actually ✨

✍️ AUTHOR

NCO manual pages written by Charlie Zender and originally formatted by Brian Mays. legends, both of them 🫶

🐛 REPORTING BUGS

Found a bug? Report bugs to <http://sf.net/bugs/?group_id=3331>. they wanna hear it bestie, don't be shy 💌

Copyright © 1995-present Charlie Zender

This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

⚠️ like, zero warranty. none. nada. it's giving "you get what you get" 💀

👀 SEE ALSO

The full documentation for NCO is maintained as a Texinfo manual called the NCO Users Guide. Now here's the thing bestie — because NCO is mathematical in nature, the documentation includes TeX-intensive portions that are just not viewable on character-based displays. So the only complete and authoritative versions of the NCO Users Guide are the PDF (recommended), DVI, and Postscript versions, at:

And the HTML and XML versions live at <http://nco.sf.net/nco.html> and <http://nco.sf.net/nco.xml> respectively 🌐

If the info and NCO programs are properly installed at your site, this command:

info nco

should give you access to the complete manual — except for those TeX-intensive portions we talked about 🙈

And the whole friend group 🫶:

ncap2(1), ncatted(1), ncbo(1), ncclimo(1), nces(1), ncecat(1), ncflint(1), ncz2psx(1), ncks(1), nco(1), ncpdq(1), ncra(1), ncrcat(1), ncremap(1), ncrename(1), ncwa(1)

🏠 HOMEPAGE

The NCO homepage at <http://nco.sf.net> contains more information. go browse, bestie 🦋

and that's a wrap on NCKS(1) 💖 this page came from the NCO distribution and it's dated 1995-present era energy, so if you're reading a build from whenever, that's your version vibe. go forth and subset your netCDF files, you absolute main character 👑✨

📦 where this page comes from