HDF(1)

📦 version 4.2.15, in epel 9

💖 HDF(1) — like, the Hierarchical Data Format library, bestie 💖

okay so this is hdf(1) from the General Commands Manual section, and it's genuinely one of those pages that ate and left no crumbs — it's a whole library, not just some lil command. grab a snack, we're going in 🍓

✨ NAME ✨

hdf — the Hierarchical Data Format library. that's it, that's the name, and honestly? iconic. 💅

🦋 SYNOPSIS 🦋

-lmfhdf -ldf -ljpeg -lz [ -lsz ]
{HDFLIBDIR}/libmfhdf.a {HDFLIBDIR}/libdf.a {JPEGLIBDIR}/libjpeg.a
{GZIPLIBDIR}/libz.a [{SZIPLIBDIR}/libsz.a]

so yeah, that's the link line situation — note that -lsz and the {SZIPLIBDIR}/libsz.a archive are both in the optional-brackets club, everyone else is mandatory. no cap. 🎀

🌈 DESCRIPTION 🌈

HDF is a multi-object file format that makes it easy to transfer all kinds of scientific data between machines and operating systems. like, cross-platform glow-up energy. ✨

wanna see which platforms are currently supported? the page says check http://www.hdfgroup.org/release4/platforms.html — that's the list.

HDF lets data self-define its own content, and it's easy to extend for future enhancements or for compatibility with other standard formats. it comes with Fortran and C calling interfaces, plus utilities for manipulating, viewing, and analyzing data in HDF files. the library contains interfaces for storing and retrieving:

  • compressed or uncompressed 8-bit and 24-bit raster images with palettes
  • n-Dimensional scientific datasets
  • binary tables

and there's also an interface that lets you group other HDF objects arbitrarily. group chat for data structures, basically. 🫶

🎨 HDF Raster Images

HDF supports storing both 8-bit and 24-bit raster images. besides storing info about the dimensions and palette of a raster image, HDF also supports raster image compression.

in previous versions of HDF (the DF interfaces, iykyk) both Run-length encoding and Imcomp compression were supported. the currently supported compression methods are:

  • simple RLE
  • N-bit
  • Skipping huffman
  • Gzip
  • Szip
  • JPEG

Imcomp compression is no longer supported — but the library can still read images that were compressed with Imcomp. so it's giving "we don't do that anymore but we remember" 😭💀

🔬 HDF Scientific Data Sets

Scientific Data Sets (SDSs) are useful for storing n-Dimensional gridded data. the actual data in the dataset can be any of the "standard" number types:

  • 8, 16 and 32bit signed and unsigned integers
  • 32 and 64bit floating point values

on top of that, a certain amount of meta-data can be stored with an SDS, including:

  • the coordinate system to use when interpreting or displaying the data
  • scales to be used for each dimension
  • labels for each dimension and the dataset as a whole
  • units for each dimension and the data
  • the valid max and min values for the data
  • calibration information for the data
  • fill or missing value information
  • ability of having more than one file open at a time
  • a more general framework for meta-data within the SDS data-model (allowing 'name = value' style meta-data)
  • support for an "unlimited dimension" in the SDS data-model, making it possible to append planes to an array along one dimension

that unlimited dimension thing? lowkey main character energy. 👑

🏷️ HDF Annotations

any object in an HDF file can have annotations associated with it. there are a number of types:

  • Labels — assumed to be short strings giving the "name" of a data object
  • Descriptions — longer text segments that are useful for giving more indepth information about a data object
  • File annotations — assumed to apply to all of the objects in a single file

🧚 HDF Vset Interfaces

the Vset module provides interfaces to two basic HDF building blocks:

  • Vgroups are generic grouping elements that let a user associate related objects within an HDF file. and because Vgroups can contain other Vgroups, you can build a hierarchical file. it's giving folder structure but make it data 🌸
  • Vdatas are data structures made up of fields and records. data is organized into "fields" within each Vdata, and each field is identified by a unique "fieldname". the type of each field may be any of the basic number types that HDF supports, and fields of different types may exist within the same Vdata. ✨

by combining Vdatas in Vgroups it's possible to represent higher level data constructs, like:

  • mesh data
  • multi-variate datasets
  • sparse matrices
  • finite-element data
  • spreadsheets
  • splines
  • non-Cartesian coordinate data
  • etc.

ok "etc." we see you 👀

🔗 HDF and netCDF

starting with HDF version 3.3, netCDF v.2.3.2 of Unidata is supported with the SD multifile interface. the SD and netCDF interfaces can read both netCDF files and multi-dimensional arrays (SDS) stored in the HDF4 files, transparently. for more info, see Chapter 3, "Scientific Data Sets", of the HDF User's Guide. 📖

wanna disable netCDF interfaces in the HDF library? configure the library using the --disable-netcdf flag and rebuild it. periodt.

🧪 EXAMPLES 🧪

all HDF routines require the header "hdf.h" to be included in the C source file — unless if the SD routines are used, in which case you should include "mfhdf.h" instead. don't mix these up bestie 🙈

Fortran programs should use:

  • "dffunc.inc" for all interfaces
  • "mffunc.inc" for the SD interfaces
  • "hdf.inc" for non-SD interfaces

to compile a program that makes HDF calls on most Unix platforms:

(FORTRAN):
{HDFLIBDIR}/bin/h4fc myprog.f

(C):
{HDFLIBDIR}/bin/h4cc myprog.c

look at us, compiling. slay. 🔥

📚 DOCUMENTATION 📚

the HDF web site lives at http://www.hdfgroup.org/.

for the vast majority of users, the "HDF User's Guide" and "HDF Reference Manual" should be sufficient. genuinely, start there 🫶

these documents can be viewed or downloaded at http://www.hdfgroup.org/products/hdf4/.

🏢 VENDOR 🏢

The HDF Group 1800 South Oak Street, Suite 203 Champaign, IL 61820 USA www.hdfgroup.org

⭐ VERSION ⭐

4.2.15

👑 LICENSE & SOURCE AVAILABILITY 👑

Copyright by The HDF Group. Copyright by the Board of Trustees of the University of Illinois.

All rights reserved.

this file is part of HDF. the full HDF copyright notice — including terms governing use, modification, and redistribution — is contained in the COPYING file, which can be found at the root of the source code distribution tree, or in https://support.hdfgroup.org/ftp/HDF/releases/.

if you don't have access to either file, you may request a copy from help@hdfgroup.org.

💌 CONTACT & HELP 💌

The HDF Group Email: help@hdfgroup.org

they literally put the email twice because they want you to use it. respect. 😌

🗂️ FILES 🗂️

  • /usr/local/lib/hdf/{libmfhdf.a,libdf.a,libjpeg.a,libz.a [,libsz.a]} — hdf libraries
  • /usr/local/bin — Location of most hdf utilities
  • /usr/local/include/hdf — Location of include file hdf.h, mfhdf.h, and others

ok that's the whole page, no gatekeeping 💜 this is THG HDF 4.2.15, dated November 2019 — mwah, go read the User's Guide, drink some water, and hydrate your data too 🧚💖

📦 where this page comes from