PAX(1P)

📦 version 2017a, in alpine 3.22, alpine 3.23, alpine 3.24, arch rolling, nixpkgs unstable, opensuse-leap 16.0

💖 PROLOG 💖

okay so before we even start, real talk: this manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ — so if you're on Linux, go peek at the corresponding Linux manual page for the deets on actual Linux behavior. Or, plot twist, the interface might just not be implemented on Linux at all. 🙈 iykyk.

✨ NAME ✨

pax — portable archive interchange 🎀

(it's giving "the one utility to rule them all" fr fr)

🌈 SYNOPSIS 🌈

pax [-dv] [-c|-n] [-H|-L] [-o options] [-f archive] [-s replstr]...
    [pattern...]

pax -r[-c|-n] [-dikuv] [-H|-L] [-f archive] [-o options]... [-p string]...
    [-s replstr]... [pattern...]

pax -w [-dituvX] [-H|-L] [-b blocksize] [[-a] [-f archive]] [-o options]...
    [-s replstr]... [-x format] [file...]

pax -r -w [-diklntuvX] [-H|-L] [-o options]... [-p string]...
    [-s replstr]... [file...] directory

four whole forms bestie. four. and yes, the spacing is load-bearing, don't touch it 💅

🦋 DESCRIPTION 🦋

So the pax utility shall read, write, and write lists of the members of archive files, and copy directory hierarchies. A whole variety of archive formats shall be supported — peep the -x format option for that tea. 👑

The action it takes depends on whether -r and -w show up. The four combinations of -r and -w are referred to as the four modes of operation: list, read, write, and copy modes — matching respectively the four forms shown up in the SYNOPSIS section. main character energy, four fits, four forms. ⭐

list — In list mode (when neither -r nor -w are specified), pax shall write the names of the members of the archive file read from the standard input, with pathnames matching the specified patterns, to standard output. If a named file is of type directory, the file hierarchy rooted at that file shall be listed as well. 🍓
read — In read mode (when -r is specified, but -w is not), pax shall extract the members of the archive file read from the standard input, with pathnames matching the specified patterns. If an extracted file is of type directory, the file hierarchy rooted at that file shall be extracted as well. The extracted files shall be created performing pathname resolution with the directory in which pax was invoked as the current working directory.

If an attempt is made to extract a directory when the directory already exists, this shall not be considered an error. If an attempt is made to extract a FIFO when the FIFO already exists, this shall not be considered an error either. (we love a forgiving queen 😌)

The ownership, access, and modification times, and file mode of the restored files are discussed under the -p option. 📁

write — In write mode (when -w is specified, but -r is not), pax shall write the contents of the file operands to the standard output in an archive format. If no file operands are specified, a list of files to copy, one per line, shall be read from the standard input and each entry in this list shall be processed as if it had been a file operand on the command line. A file of type directory shall include all of the files in the file hierarchy rooted at the file. ✍️
copy — In copy mode (when both -r and -w are specified), pax shall copy the file operands to the destination directory.

If no file operands are specified, a list of files to copy, one per line, shall be read from the standard input. A file of type directory shall include all of the files in the file hierarchy rooted at the file.

The effect of the copy shall be as if the copied files were written to a pax format archive file and then subsequently extracted — except that copying of sockets may be supported even if archiving them in write mode is not supported, and that there may be hard links between the original and the copied files. If the destination directory is a subdirectory of one of the files to be copied, the results are unspecified 😭. If the destination directory is a file of a type not defined by the System Interfaces volume of POSIX.1‐2017, the results are implementation-defined; otherwise, it shall be an error for the file named by the directory operand not to exist, not be writable by the user, or not be a file of type directory.

In read or copy modes, if intermediate directories are necessary to extract an archive member, pax shall perform actions equivalent to the mkdir() function defined in the System Interfaces volume of POSIX.1‐2017, called with the following arguments:

The intermediate directory used as the path argument
The value of the bitwise-inclusive OR of S_IRWXU, S_IRWXG, and S_IRWXO as the mode argument

If any specified pattern or file operands are not matched by at least one file or archive member, pax shall write a diagnostic message to standard error for each one that did not match and exit with a non-zero exit status. 🚨 (no silent failures in this house)

The archive formats described in the EXTENDED DESCRIPTION section shall be automatically detected on input. The default output archive format shall be implementation-defined.

A single archive can span multiple files. The pax utility shall determine, in an implementation-defined manner, what file to read or write as the next file.

If the selected archive format supports the specification of linked files, it shall be an error if these files cannot be linked when the archive is extracted. For archive formats that do not store file contents with each name that causes a hard link, if the file that contains the data is not extracted during this pax session, either the data shall be restored from the original file, or a diagnostic message shall be displayed with the name of a file that can be used to extract the data. In traversing directories, pax shall detect infinite loops — that is, entering a previously visited directory that is an ancestor of the last file visited. When it detects an infinite loop, pax shall write a diagnostic message to standard error and shall terminate. 💀 (we do not do infinite loops here, bestie, we log off)

💅 OPTIONS 💅

The pax utility shall conform to the Base Definitions volume of POSIX.1‐2017, Section 12.2, Utility Syntax Guidelines — except that the order of presentation of the -o, -p, and -s options is significant. so like, don't shuffle those, bestie, order matters 🙏

The following options shall be supported:

-r — Read an archive file from standard input.
-w — Write files to the standard output in the specified archive format.
-a — Append files to the end of the archive. It is implementation-defined which devices on the system support appending. Additional file formats unspecified by this volume of POSIX.1‐2017 may impose restrictions on appending. (appending is not a universal right, okurr)
-b blocksize — Block the output at a positive decimal integer number of bytes per write to the archive file. Devices and archive formats may impose restrictions on blocking. Blocking shall be automatically determined on input. Conforming applications shall not specify a blocksize value larger than 32256. Default blocking when creating archives depends on the archive format. (See the -x option below.) 📏
-c — Match all file or archive members except those specified by the pattern or file operands.
-d — Cause files of type directory being copied or archived, or archive members of type directory being extracted or listed, to match only the file or archive member itself and not the file hierarchy rooted at the file.
-f archive — Specify the pathname of the input or output archive, overriding the default standard input (in list or read modes) or standard output (write mode).
-H — If a symbolic link referencing a file of type directory is specified on the command line, pax shall archive the file hierarchy rooted in the file referenced by the link, using the name of the link as the root of the file hierarchy. Otherwise, if a symbolic link referencing a file of any other file type which pax can normally archive is specified on the command line, then pax shall archive the file referenced by the link, using the name of the link. The default behavior, when neither -H or -L are specified, shall be to archive the symbolic link itself. 🔗
-i — Interactively rename files or archive members. For each archive member matching a pattern operand or file matching a file operand, a prompt shall be written to the file /dev/tty. The prompt shall contain the name of the file or archive member, but the format is otherwise unspecified. A line shall then be read from /dev/tty. If this line is blank, the file or archive member shall be skipped. If this line consists of a single period, the file or archive member shall be processed with no modification to its name. Otherwise, its name shall be replaced with the contents of the line. The pax utility shall immediately exit with a non-zero exit status if end-of-file is encountered when reading a response or if /dev/tty cannot be opened for reading and writing.

The results of extracting a hard link to a file that has been renamed during extraction are unspecified. 😭

-k — Prevent the overwriting of existing files. 🛑
-l — (The letter ell.) In copy mode, hard links shall be made between the source and destination file hierarchies whenever possible. If specified in conjunction with -H or -L, when a symbolic link is encountered, the hard link created in the destination file hierarchy shall be to the file referenced by the symbolic link. If specified when neither -H nor -L is specified, when a symbolic link is encountered, the implementation shall create a hard link to the symbolic link in the source file hierarchy or copy the symbolic link to the destination. 🫶
-L — okay so if a symbolic link that points at a directory shows up on the command line, or gets stumbled upon while walking a file hierarchy, pax is gonna archive the whole file hierarchy rooted in the file the link points to, and it uses the name of the link as the root of that hierarchy. 💫 Otherwise? If a symbolic link points at literally any other file type that pax can normally archive, and it shows up on the command line or during traversal, pax archives the file the link references, using the name of the link. And the default behavior — when neither -H nor -L is given — is to archive the symbolic link itself. no cap.
-n — this one picks the first archive member that matches each pattern operand. Not more than one archive member gets matched per pattern (though members of type directory are still gonna match the file hierarchy rooted at that file). one and done bestie 💅
-o options — this hands info to the implementation so it can modify how files get extracted or written. The value of options is one or more <comma>-separated keywords, shaped like this:

keyword[[:]=value][,keyword[[:]=value], ...]

Some keywords only apply to certain file formats, and each description will tell you which. Using keywords that don't apply to the file format you're processing = undefined results, so like… don't. 🙈

Keywords in the options argument have to be a string that would be a valid portable filename, as described in the Base Definitions volume of POSIX.1‐2017, Section 3.282, Portable Filename Character Set.

Note: keywords aren't expected to be filenames, they just follow the same character composition rules as portable filenames.

Keywords can have white space in front of them. The value field is zero or more characters; inside value, the application has to put a <backslash> before any literal <comma>, and that backslash gets ignored, but it preserves the <comma> as part of value. A <comma> as the final character, or a <comma> followed only by white space at the end, in options — ignored. You can stack multiple -o options; if the keywords given to those multiple -o options conflict, the keywords and values that appear later in command line sequence win, and the earlier ones get silently ignored. 👑

These keyword values of options shall be supported for the file formats as indicated:

delete=pattern

(Applicable only to the -x pax format.) When used in write or copy mode, pax shall omit from extended header records that it produces any keywords matching the string pattern. When used in read or list mode, pax shall ignore any keywords matching the string pattern in the extended header records. In both cases, matching uses the pattern matching notation described in Section 2.13.1, Patterns Matching a Single Character and Section 2.13.2, Patterns Matching Multiple Characters. For example:

-o delete=security.*

would suppress security-related information. See pax Extended Header for extended header record keyword usage.

When multiple -odelete=pattern options are specified, the patterns are additive; all keywords matching the specified string patterns get omitted from extended header records that pax produces.

exthdr.name=string

(Applicable only to the -x pax format.) This keyword lets the user control the name written into the ustar header blocks for the extended header produced under the circumstances described in pax Header Block. The name shall be the contents of string, after these character substitutions have been made:

%d — the directory name of the file, equivalent to the result of the dirname utility on the translated pathname.
%f — the filename of the file, equivalent to the result of the basename utility on the translated pathname.
%p — the process ID of the pax process.
%% — a % character.

Any other % characters in string produce undefined results. 😬

If no -o exthdr.name=string is specified, pax shall use the following default value:

%d/PaxHeaders.%p/%f

globexthdr.name=string

(Applicable only to the -x pax format.) When used in write or copy mode with the appropriate options, pax shall create global extended header records with ustar header blocks that will be treated as regular files by previous versions of pax. This keyword lets the user control the name written into the ustar header blocks for global extended header records. The name shall be the contents of string, after these character substitutions have been made:

%n — an integer that represents the sequence number of the global extended header record in the archive, starting at 1.
%p — the process ID of the pax process.
%% — a % character.

Any other % characters in string produce undefined results. 🫠

If no -o globexthdr.name=string is specified, pax shall use the following default value:

$TMPDIR/GlobalHead.%p.%n

where $TMPDIR represents the value of the TMPDIR environment variable. If TMPDIR is not set, pax shall use /tmp.

invalid=action

(Applicable only to the -x pax format.) This keyword lets the user control what pax does when it encounters values in an extended header record that — in read or copy mode — are invalid in the destination hierarchy, or, in list mode, can't be written in the codeset and current locale of the implementation. These are the invalid values pax shall recognize:

In read or copy mode, a filename or link name containing character encodings that are invalid in the destination hierarchy. (Like, the name might have embedded NULs. 💀)
In read or copy mode, a filename or link name that's longer than the max allowed in the destination hierarchy (either for a pathname component or the entire pathname).
In list mode, any character string value (filename, link name, user name, and so on) that can't be written in the codeset and current locale of the implementation.

These mutually-exclusive values of the action argument are supported:

binary — in write mode, pax shall generate a hdrcharset=BINARY extended header record for each file with a filename, link name, group name, owner name, or any other field in an extended header record that cannot be translated to the UTF‐8 codeset, which lets the archive contain the files with unencoded extended header record values. In read or copy mode, pax shall use the values specified in the header without translation, regardless of whether that might overwrite an existing file with a valid name. In list mode, pax shall behave identically to the bypass action.

bypass — bestie, in read or copy mode pax just... skips the file entirely, no change to the destination hierarchy at all, it's giving ghosting 💅. In list mode though, pax writes all the requested valid values for the file, but its method for writing invalid values is unspecified (mysterious queen behavior, we don't ask).
rename — in read or copy mode, pax acts like the -i option were in effect for every file with invalid filename or link name values, so the user can hand over a replacement name interactively, very "let me fix this for you" 🫶. In list mode, it behaves identically to the bypass action. Same energy, different mode.
UTF‐8 — when this is used in read, copy, or list mode and a filename, link name, owner name, or any other field in an extended header record literally cannot be translated from the pax UTF‐8 codeset format to the codeset and current locale of the implementation, pax uses the actual UTF‐8 encoding for the name instead. If a hdrcharset extended header record is in effect for this file, the character set specified by that record gets used instead of UTF‐8. And if a hdrcharset=BINARY extended header record is in effect for this file, no translation shall be performed. Periodt, zero translation, we respect the binary 🧚.
write — in read or copy mode, pax writes the file, translating the name, regardless of whether this may overwrite an existing file with a valid name. It said what it said 💀. In list mode, pax behaves identically to the bypass action.

If no -o invalid=option is specified, pax acts as if -oinvalid=bypass were specified. Any overwriting of existing files that may be allowed by the -oinvalid= actions is subject to permission (-p) and modification time (-u) restrictions, and gets suppressed if the -k option is also specified. So many rules but ok, we love a system with boundaries ✨.

`linkdata` 🎀

Applicable only to the -x pax format, just so you know.

In write mode, pax writes the contents of a file to the archive even when that file is merely a hard link to a file whose contents have already been written to the archive. Yes bestie, it's duplicating on purpose, that's the whole point 💖.

`listopt=format` 🌸

This keyword specifies the output format of the table of contents produced when the -v option is specified in list mode. See List Mode Format Specifications. To avoid ambiguity, the listopt=format shall be the only or final keyword=value pair in a -o option-argument — all characters in the remainder of the option-argument are considered part of the format string, no take-backs 🫶. When multiple -olistopt=format options are specified, the format strings are considered a single, concatenated string, evaluated in command line order. So order matters, keep up 🦋.

`times` ⭐

Applicable only to the -x pax format.

When used in write or copy mode, pax includes atime and mtime extended header records for each file. See pax Extended Header File Times. Timestamps stay timestamping, no cap 💅.

In addition to these keywords, if the -x pax format is specified, any of the keywords and values defined in pax Extended Header, including implementation extensions, can be used in -o option-arguments, in either of two modes:

keyword=value — when used in write or copy mode, these keyword/value pairs are included at the beginning of the archive as typeflag g global extended header records. When used in read or list mode, these keyword/value pairs act as if they had been at the beginning of the archive as typeflag g global extended header records. Global vibes only ✨.
keyword:=value — when used in write or copy mode, these keyword/value pairs are included as records at the beginning of a typeflag x extended header for each file. (This is equivalent to the <equals-sign> form except that it creates no typeflag g global extended header records.) When used in read or list mode, these keyword/value pairs act as if they were included as records at the end of each extended header; thus, they override any global or file-specific extended header record keywords of the same names. The way that this one just overrides everything, main character energy fr fr 👑. For example, in the command:

pax -r -o "
gname:=mygroup,
" <archive

the group name will be forced to a new value for all files read from the archive. Forced, bestie, no negotiation 💅.

The precedence of -o keywords over various fields in the archive is described in pax Extended Header Keyword Precedence. If the -o delete=pattern, -o keyword=value, or -o keyword:=value options are used to override or remove any extended header data needed to find files in an archive (e.g., -o delete=size for a file whose size cannot be represented in a ustar header or -o size=100 for a file whose size is not 100 bytes), the behavior is undefined.

⚠️ Undefined behavior, bestie. That means we do not do this. Not even for science. Do NOT test it 😭.

`-p string` 💖

Specify one or more file characteristic options (privileges). The string option-argument shall be a string specifying file characteristics to be retained or discarded on extraction. The string consists of the specification characters a, e, m, o, and p. Other implementation-defined characters can be included. Multiple characteristics can be concatenated within the same string and multiple -p options can be specified. The meaning of the specification characters are as follows:

a — do not preserve file access times. Bye atime 👋.
e — preserve the user ID, group ID, file mode bits (see the Base Definitions volume of POSIX.1‐2017, Section 3.169, File Mode Bits), access time, modification time, and any other implementation-defined file characteristics. It's giving "keep everything, I'm sentimental" 🥹.
m — do not preserve file modification times. Not the mtimes 😭.
o — preserve the user ID and group ID. Ownership stays iconic 💅.
p — preserve the file mode bits. Other implementation-defined file mode attributes may be preserved too. Permission glow-up preserved ✨.

In the preceding list, ``preserve'' indicates that an attribute stored in the archive shall be given to the extracted file, subject to the permissions of the invoking process. The access and modification times of the file are preserved unless otherwise specified with the -p option or not stored in the archive. All attributes that are not preserved shall be determined as part of the normal file creation action (see Section 1.1.1.4, File Read, Write, and Creation). Default behavior, we love her.

If neither the e nor the o specification character is specified, or the user ID and group ID are not preserved for any reason, pax shall not set the S_ISUID and S_ISGID bits of the file mode.

If the preservation of any of these items fails for any reason, pax shall write a diagnostic message to standard error. Failure to preserve these items shall affect the final exit status, but shall not cause the extracted file to be deleted. So like, it'll be messy and complain about it, but it won't nuke your file 🙈.

If file characteristic letters in any of the string option-arguments are duplicated or conflict with each other, the ones given last shall take precedence. For example, if -p eme is specified, file modification times are preserved. Last one wins, that's the rule, periodt 💅.

`-s replstr` 🦋

Modify file or archive member names named by pattern or file operands according to the substitution expression replstr, using the syntax of the ed utility. The concepts of address'' and line'' are meaningless in the context of the pax utility, and shall not be supplied. The format shall be:

-s /old/new/[gp]

where as in ed, old is a basic regular expression and new can contain an <ampersand>, '\n' (where n is a digit) back-references, or subexpression matching. The old string is also permitted to contain <newline> characters. Newlines in there? Sure, go off 🎀.

Any non-null character can be used as a delimiter ('/' shown here). Multiple -s expressions can be specified; the expressions are applied in the order specified, terminating with the first successful substitution. The optional trailing 'g' is as defined in the ed utility. The optional trailing 'p' shall cause successful substitutions to be written to standard error. File or archive member names that substitute to the empty string shall be ignored when reading and writing archives. Empty string = ignored, we don't see her 💅.

`-t` 🌈

When reading files from the file system, and if the user has the permissions required by utime() to do so, set the access time of each file read to the access time that it had before being read by pax. Putting the atime back exactly where it was, a true restoration icon 👑.

-u — bestie, this one says "skip the old stuff." Like, ignore files that are older (meaning their file modification time is less recent) than a file or archive member with the same name that already exists. Let me break it down by mode, because the vibes differ:
- read mode: an archive member with the same name as a file in the file system gets extracted if the archive member is newer than the file.
- write mode: an archive file member with the same name as a file in the file system gets superseded if the file is newer than the archive member. If -a is also specified, this happens by appending to the archive; otherwise, whether it's done by actual replacement in the archive or by appending to the archive is unspecified — like, nobody's promising you which one, okay?
- copy mode: the file in the destination hierarchy gets replaced by the file in the source hierarchy, or by a link to the file in the source hierarchy, if the file in the source hierarchy is newer. 💅
-v — the chatty one 🗣️. In list mode, it produces a verbose table of contents (peep the STDOUT section for that). Otherwise, it writes archive member pathnames to standard error (that's the STDERR section's business).
-x format — pick your output archive format, bestie. The pax utility has to support these:
- cpio — the cpio interchange format; check the EXTENDED DESCRIPTION section. The default blocksize for this format for character special archive files is 5120. Implementations have to support all blocksize values less than or equal to 32256 that are multiples of 512.
- pax — the pax interchange format; check the EXTENDED DESCRIPTION section. The default blocksize for this format for character special archive files is 5120. Implementations have to support all blocksize values less than or equal to 32256 that are multiples of 512.
- ustar — the tar interchange format; check the EXTENDED DESCRIPTION section. The default blocksize for this format for character special archive files is 10240. Implementations have to support all blocksize values less than or equal to 32256 that are multiples of 512.

Implementation-defined formats have to specify a default block size, plus any other block sizes they support, for character special archive files. 🎀

Real talk: any attempt to append to an archive file in a format different from the existing archive format makes pax exit immediately with a non-zero exit status. No discussion. Periodt. 💀

-X — when you're traversing the file hierarchy that a pathname specifies, pax shall not descend into directories that have a different device ID (st_dev; see the System Interfaces volume of POSIX.1‐2017, stat()). She stays in her lane. 🚧

Specifying more than one of the mutually-exclusive options -H and -L shall not be considered an error, and the last option specified determines the behavior of the utility. So like, whoever speaks last wins. ✨

🌸 how the name-y options all vibe together 🌸

The options that operate on the names of files or archive members (-c, -i, -n, -s, -u, and -v) interact like this:

In read mode: the archive members get selected based on the user-specified pattern operands as modified by the -c, -n, and -u options. Then any -s and -i options modify the names of the selected files, in that order. The -v option writes the names that result from these modifications. 🦋

In write mode: the files get selected based on the user-specified pathnames as modified by the -n and -u options. Then any -s and -i options modify the names of these selected files, in that order. The -v option writes the names that result from these modifications. 💜

If both -u and -n are specified, pax shall not consider a file selected unless it is newer than the file it's being compared to.

🎀 List Mode Format Specifications 🎀

In list mode with the -o listopt=format option, the format argument gets applied for each selected file. The pax utility appends a <newline> to the listopt output for each selected file. The format argument is used as the format string described in the Base Definitions volume of POSIX.1‐2017, Chapter 5, File Format Notation — with exceptions 1. through 6. defined in the EXTENDED DESCRIPTION section of printf, plus these following exceptions:

The sequence (keyword) can occur before a format conversion specifier. The conversion argument is defined by the value of keyword. The implementation shall support these keywords:
- Any of the Field Name entries in Table 4-14, ustar Header Block and Table 4-16, Octet-Oriented cpio Archive Entry. The implementation may support the cpio keywords without the leading c_ in addition to the form required by Table 4-16, Octet-Oriented cpio Archive Entry.
- Any keyword defined for the extended header in pax Extended Header.
- Any keyword provided as an implementation-defined extension within the extended header defined in pax Extended Header.

For example, the sequence "%(charset)s" is the string value of the name of the character set in the extended header.

The result of the keyword conversion argument is the value from the applicable header field or extended header, without any trailing NULs.

All keyword values used as conversion arguments get translated from the UTF‐8 encoding (or whatever alternative encoding is specified by any hdrcharset extended header record) to the character set appropriate for the local file system, user database, etc., as applicable. 🌈

An additional conversion specifier character, T, specifies time formats. The T conversion specifier character can be preceded by the sequence (keyword=subformat), where subformat is a date format as defined by date operands. The default keyword is mtime and the default subformat is:

%b %e %H:%M %Y

An additional conversion specifier character, M, specifies the file mode string as defined in ls Standard Output. If (keyword) is omitted, the mode keyword is used. For example, %.1M writes the single character corresponding to the <entry type> field of the ls -l command.
An additional conversion specifier character, D, specifies the device for block or special files, if applicable, in an implementation-defined format. If it's not applicable, and (keyword) is specified, then this conversion is equivalent to %(keyword)u. If it's not applicable, and (keyword) is omitted, then this conversion is equivalent to <space>.
An additional conversion specifier character, F, specifies a pathname. The F conversion character can be preceded by a sequence of <comma>-separated keywords:

(keyword[,keyword] ... )

The values for all the keywords that are non-null get concatenated together, each separated by a /. The default is (path) if the keyword path is defined; otherwise, the default is (prefix,name).

An additional conversion specifier character, L, specifies a symbolic link expansion. If the current file is a symbolic link, then %L expands to:

"%s -> %s", <value of keyword>, <contents of link>

Otherwise, the %L conversion specification is the equivalent of %F. 🔗

👑 OPERANDS 👑

These operands shall be supported:

directory — the destination directory pathname for copy mode. 📁
file — a pathname of a file to be copied or archived. 📄
pattern — a pattern matching one or more pathnames of archive members. A pattern must be given in the name-generating notation of the pattern matching notation in Section 2.13, Pattern Matching Notation, including the filename expansion rules in Section 2.13.3, Patterns Used for Filename Expansion. The default, if no pattern is specified, is to select all members in the archive. 🎯

💖 STDIN 💖

In write mode, the standard input shall be used only if no file operands are specified. It shall be a file containing a list of pathnames, each terminated by a <newline> character.

In list and read modes, if -f is not specified, the standard input shall be an archive file.

Otherwise, the standard input shall not be used. 🙈

🧚 INPUT FILES 🧚

The input file named by the archive option-argument — or standard input when the archive is read from there — shall be a file formatted according to one of the specifications in the EXTENDED DESCRIPTION section, or some other implementation-defined format.

The file /dev/tty shall be used to write prompts and read responses. ⌨️

🌸 ENVIRONMENT VARIABLES 🌸

okay bestie, these are the little gremlins in your shell that quietly change how pax behaves. they said "we're gonna affect the execution" and they meant it 💅

LANG — gives a default value for all the internationalization variables that are unset or null. (this is from the Base Definitions volume of POSIX.1‐2017, Section 8.2, Internationalization Variables — that's where the precedence rules for all these locale things live, iykyk)
LC_ALL — if you set it to a non-empty string value, it straight-up overrides every other internationalization variable. main character energy, honestly 👑
LC_COLLATE — sets the locale for how ranges, equivalence classes, and multi-character collating elements behave. where do those show up? in the pattern matching expressions for the pattern operand, in the basic regular expression for the -s option, and in the extended regular expression defined for the yesexpr locale keyword over in the LC_MESSAGES category. so many layers fr fr
LC_CTYPE — sets the locale for reading sequences of bytes as characters (think single-byte vs multi-byte characters in arguments and input files 🦋), for how character classes behave in the extended regular expression defined for the yesexpr locale keyword in LC_MESSAGES, and for pattern matching. busy queen
LC_MESSAGES — sets the locale used to process affirmative responses, plus the locale that shapes the format and contents of diagnostic messages and prompts written to standard error. this is the one that decides how your errors sound 😭
LC_TIME — sets the format and contents of date and time strings whenever you use the -v option ⏰
NLSPATH — tells pax where to find the message catalogs for processing LC_MESSAGES.
TMPDIR — gives the pathname that makes up part of the default global extended header record file — exactly like it's described for the -o globexthdr= keyword back in the OPTIONS section. remember that one?
TZ — sets the timezone used to calculate date and time strings when you pass -v. if TZ is unset or null, an unspecified default timezone gets used. mystery timezone ✨

⚡ ASYNCHRONOUS EVENTS ⚡

Default.

(yes that's the whole section. she said "no thoughts, head empty" and honestly? valid 🎀)

💖 STDOUT 💖

write mode

if -f is not specified, standard output is the archive. it gets formatted according to one of the specs over in the EXTENDED DESCRIPTION section, or some other implementation-defined format (check out -x format). so like, pipe it somewhere nice 🫶

list mode

when -olistopt=format has been specified, the selected archive members get written to standard output using the format described under List Mode Format Specifications.

but in list mode without the -olistopt=format option, the table of contents of the selected archive members goes to standard output in this format:

"%s\n", <pathname>

and if you add -v in list mode, that table of contents gets written to standard output using these formats instead:

for pathnames that represent hard links to previous members of the archive:

"%s == %s\n", <ls -l listing>, <linkname>

for all other pathnames:

"%s\n", <ls -l listing>

where <ls -l listing> is the format the ls utility spits out with its -l option. and when pax writes pathnames in this format, it is unspecified what gets written for fields where the underlying archive format doesn't actually have the correct info — though the correct number of <blank>-separated fields shall be written. so it's giving filled-in-but-maybe-lies 💀

one more thing: in list mode, standard output shall not be buffered more than a pathname (plus any associated info and a <newline> terminator) at a time. no hoarding, queen.

🔥 STDERR 🔥

if -v is specified in read, write, or copy modes, pax writes the pathnames it processes to standard error output like this:

"%s\n", <pathname>

these pathnames get written as soon as processing begins on the file or archive member, and they get flushed to standard error. the trailing <newline> — which shall not be buffered — is written when the file has been read or written. timing is everything ✨

if the -s option is specified and the replacement string has a trailing 'p', substitutions get written to standard error in this format:

"%s >> %s\n", <original pathname>, <new pathname>

and in all operating modes of pax, optional messages of unspecified format may be written to standard error — stuff about the input archive format and volume number, the number of files, blocks, volumes, and media parts, plus other diagnostic messages. so if stderr is chatty, that's why 🙈

in all formats, for both standard output and standard error, it is unspecified how non-printable characters in pathnames or link names get written. no promises there bestie.

okay this one matters: when you're using the -xpax archive format, if a filename, link name, group name, owner name, or any other field in an extended header record can't be translated between the codeset in use for that extended header record and the character set of the current locale, pax shall write a diagnostic message to standard error, shall process the file as described for the -o invalid= option, and then shall keep on going with the next file. it doesn't crash out, it just ✨notes it and moves on✨

🦋 OUTPUT FILES 🦋

in read mode, the extracted output files shall be of the archived file type. in copy mode, the copied output files shall be the type of the file being copied. either way, existing files in the destination hierarchy get overwritten only when all three of these tests allow it:

the permission test (-p)
the modification time test (-u)
the invalid-value test (-oinvalid=)

three locks, all gotta open 🔐

in write mode, the output file named by the -f option-argument shall be a file formatted according to one of the specifications in the EXTENDED DESCRIPTION section, or some other implementation-defined format.

🌈 EXTENDED DESCRIPTION 🌈

okay this is the deep lore section. buckle up, we're going in 🧚

🎀 pax Interchange Format 🎀

a pax archive tape or file produced in the -xpax format shall contain a series of blocks. the physical layout shall be identical to the ustar format described in ustar Interchange Format. every file archived gets represented by this sequence:

an optional header block with extended header records. this header block is of the form described in pax Header Block, with a typeflag value of x or g. the extended header records — described in pax Extended Header — shall be included as the data for this header block.
a header block that describes the file. any fields in the preceding optional extended header shall override the associated fields in this header block for this file. override queen 👑
zero or more blocks that contain the contents of the file.

at the end of the archive file there shall be two 512-byte blocks filled with binary zeros, interpreted as an end-of-archive indicator. that's the little "we're done here" 💅

a schematic of an example archive with global extended header records and two actual files is shown in Figure 4-1, pax Format Archive Example. in the example, the second file in the archive has no extended header preceding it — presumably because it has no need for extended attributes. low maintenance file, we love her.

Figure 4-1: pax Format Archive Example

✨ pax Header Block ✨

the pax header block shall be identical to the ustar header block described in ustar Interchange Format, except that two additional typeflag values are defined:

x — represents extended header records for the following file in the archive (which shall have its own ustar header block). the format of these extended header records shall be as described in pax Extended Header.
g — represents global extended header records for the following files in the archive. the format of these extended header records shall be as described in pax Extended Header. each value shall affect all subsequent files that don't override that value in their own extended header record, and it keeps affecting them until another global extended header record is reached that provides another value for the same field.

heads up fr fr: the typeflag g global headers should not be used with interchange media that could suffer partial data loss in transporting the archive. if the archive gets partially eaten, those global vibes are gone for everybody 😭

for both of these types, the size field shall be the size of the extended header records in octets. the other fields in the header block are not meaningful to this version of the pax utility. however — and this is the fun part — if this archive is read by a pax utility conforming to the ISO POSIX‐2:1993 standard, those header block fields are used to create a regular file containing the extended header records as data. so you should pick header block field values that give reasonable file access to that regular file. considerate king behavior 🫶

a further difference from the ustar header block: data blocks for files of typeflag 1 (the digit one) (hard link) may be included, which means the size field may be greater than zero. archives created by pax -o linkdata shall include these data blocks with the hard links. so hard links can carry luggage now 💅

💜 pax Extended Header 💜

a pax extended header contains values that are inappropriate for the ustar header block because of limitations in that format. which limitations, you ask?

fields requiring a character encoding other than the one described in the ISO/IEC 646:1991 standard
fields representing file attributes not described in the ustar header
fields whose format or length don't fit the requirements of the ustar header

the values in an extended header add attributes to the following file (or files — see the description of the typeflag g header block) or override values in the following header block(s), as indicated in the list of keywords below.

an extended header shall consist of one or more records, each constructed like this:

"%d %s=%s\n", <length>, <keyword>, <value>

the extended header records shall be encoded according to the ISO/IEC 10646‐1:2000 standard UTF‐8 encoding. the <length> field, <blank>, <equals-sign>, and <newline> shown shall be limited to the portable character set, as encoded in UTF‐8. the <keyword> fields can be any UTF‐8 characters.

the <length> field shall be the decimal length of the extended header record in octets, including the trailing <newline>. (don't forget the newline, she counts 🎀)

if there's a hdrcharset extended header in effect for a file, the value field for any gname, linkpath, path, and uname extended header records shall be encoded using the character set specified by the hdrcharset extended header record; otherwise, the value field shall be encoded using UTF‐8. the value field for all other keywords specified by POSIX.1‐2008 shall be encoded using UTF‐8.

okay so bestie, the <keyword> field shall be one of the entries from this list — or a keyword that's provided as an implementation extension, no cap. 💅

Keywords that are entirely lowercase letters, digits, and periods? Those are reserved for future standardization, periodt. And a keyword shall not include an <equals-sign>, so don't even try it. 🚫

(Also quick note: in the list below, when you see file(s) or block(s), it's just acknowledging that a keyword affects the following single file after a typeflag x extended header, but possibly multiple files after typeflag g. Any requirements for pax to include a record when in write or copy mode only apply if that record hasn't already been provided via the -o option. And when used in copy mode, pax shall behave as if an archive had been created with the applicable extended header records and then extracted. Got it? Good. 💖)

atime — The file access time for the following file(s), equivalent to the value of the st_atime member of the stat structure for a file, as described by the stat() function. The access time shall be restored if the process has the appropriate privileges required to do so. The format of the <value> shall be as described in pax Extended Header File Times. 🕰️
charset — The name of the character set used to encode the data in the following file(s). The entries in the following table are defined to refer to known standards; additional names may be agreed on between the originator and recipient.
- ISO-IR 646 1990 — ISO/IEC 646:1990
- ISO-IR 8859 1 1998 — ISO/IEC 8859‐1:1998
- ISO-IR 8859 2 1999 — ISO/IEC 8859‐2:1999
- ISO-IR 8859 3 1999 — ISO/IEC 8859‐3:1999
- ISO-IR 8859 4 1998 — ISO/IEC 8859‐4:1998
- ISO-IR 8859 5 1999 — ISO/IEC 8859‐5:1999
- ISO-IR 8859 6 1999 — ISO/IEC 8859‐6:1999
- ISO-IR 8859 7 1987 — ISO/IEC 8859‐7:1987
- ISO-IR 8859 8 1999 — ISO/IEC 8859‐8:1999
- ISO-IR 8859 9 1999 — ISO/IEC 8859‐9:1999
- ISO-IR 8859 10 1998 — ISO/IEC 8859‐10:1998
- ISO-IR 8859 13 1998 — ISO/IEC 8859‐13:1998
- ISO-IR 8859 14 1998 — ISO/IEC 8859‐14:1998
- ISO-IR 8859 15 1999 — ISO/IEC 8859‐15:1999
- ISO-IR 10646 2000 — ISO/IEC 10646:2000
- ISO-IR 10646 2000 UTF-8 — ISO/IEC 10646, UTF-8 encoding
- BINARY — None.

The encoding is included in an extended header for information only; when pax is used as described in POSIX.1‐2008, it shall not translate the file data into any other encoding. The BINARY entry indicates unencoded binary data.

When used in write or copy mode, it is implementation-defined whether pax includes a charset extended header record for a file. 🤷‍♀️

comment — A series of characters used as a comment. All characters in the <value> field shall be ignored by pax. 💬
gid — The group ID of the group that owns the file, expressed as a decimal number using digits from the ISO/IEC 646:1991 standard. This record shall override the gid field in the following header block(s). When used in write or copy mode, pax shall include a gid extended header record for each file whose group ID is greater than 2097151 (octal 7777777). 👑
gname — The group of the file(s), formatted as a group name in the group database. This record shall override the gid and gname fields in the following header block(s), and any gid extended header record. When used in read, copy, or list mode, pax shall translate the name from the encoding in the header record to the character set appropriate for the group database on the receiving system. If any of the characters cannot be translated, and if neither the -oinvalid=UTF‐8 option nor the -oinvalid=binary option is specified, the results are implementation-defined. When used in write or copy mode, pax shall include a gname extended header record for each file whose group name cannot be represented entirely with the letters and digits of the portable character set. 🫶
hdrcharset — The name of the character set used to encode the value field of the gname, linkpath, path, and uname pax extended header records. The entries in the following table are defined to refer to known standards; additional names may be agreed between the originator and the recipient.
- ISO-IR 10646 2000 UTF-8 — ISO/IEC 10646, UTF-8 encoding
- BINARY — None.

If no hdrcharset extended header record is specified, the default character set used to encode all values in extended header records shall be the ISO/IEC 10646‐1:2000 standard UTF‐8 encoding.

The BINARY entry indicates that all values recorded in extended headers for affected files are unencoded binary data from the underlying system. 🧚

linkpath — The pathname of a link being created to another file, of any type, previously archived. This record shall override the linkname field in the following ustar header block(s). The following ustar header block shall determine the type of link created. If typeflag of the following header block is 1, it shall be a hard link. If typeflag is 2, it shall be a symbolic link and the linkpath value shall be the contents of the symbolic link. The pax utility shall translate the name of the link (contents of the symbolic link) from the encoding in the header to the character set appropriate for the local file system. When used in write or copy mode, pax shall include a linkpath extended header record for each link whose pathname cannot be represented entirely with the members of the portable character set other than NUL. 🔗
mtime — The file modification time of the following file(s), equivalent to the value of the st_mtime member of the stat structure for a file, as described in the stat() function. This record shall override the mtime field in the following header block(s). The modification time shall be restored if the process has the appropriate privileges required to do so. The format of the <value> shall be as described in pax Extended Header File Times. ⏰
path — The pathname of the following file(s). This record shall override the name and prefix fields in the following header block(s). The pax utility shall translate the pathname of the file from the encoding in the header to the character set appropriate for the local file system.

When used in write or copy mode, pax shall include a path extended header record for each file whose pathname cannot be represented entirely with the members of the portable character set other than NUL. 📁

realtime.any — The keywords prefixed by realtime. are reserved for future standardization. ⏳
security.any — The keywords prefixed by security. are reserved for future standardization. 🔒
size — The size of the file in octets, expressed as a decimal number using digits from the ISO/IEC 646:1991 standard. This record shall override the size field in the following header block(s). When used in write or copy mode, pax shall include a size extended header record for each file with a size value greater than 8589934591 (octal 77777777777). 📏
uid — The user ID of the file owner, expressed as a decimal number using digits from the ISO/IEC 646:1991 standard. This record shall override the uid field in the following header block(s). When used in write or copy mode, pax shall include a uid extended header record for each file whose owner ID is greater than 2097151 (octal 7777777). 🆔
uname — The owner of the following file(s), formatted as a user name in the user database. This record shall override the uid and uname fields in the following header block(s), and any uid extended header record. When used in read, copy, or list mode, pax shall translate the name from the encoding in the header record to the character set appropriate for the user database on the receiving system. If any of the characters cannot be translated, and if neither the -oinvalid=UTF‐8 option nor the -oinvalid=binary option is specified, the results are implementation-defined. When used in write or copy mode, pax shall include a uname extended header record for each file whose user name cannot be represented entirely with the letters and digits of the portable character set. 🧑‍💻

Okay so a <value> field with zero length? That is not an empty string, bestie, that is a delete order. 💀 It deletes any header block field, any previously entered extended header value, or any global extended header value with the same name. Just absolutely gone, no trace, we don't know her.

And if a keyword in an extended header record (or in an -o option-argument — same vibe) overrides or deletes a corresponding field in the ustar header block, then pax shall ignore the contents of that header block field. Like, that field is not even in the group chat anymore. 🙈

Also, huge one, the thing everyone gets wrong: unlike the ustar header block fields, NULs shall not delimit <value>s. Every single character inside the <value> field counts as data for that field. And none of the length limitations of the ustar header block fields in Table 4-14, ustar Header Block apply to the extended header records. No cap, no limit, it's giving unbothered. ✨

💫 pax Extended Header Keyword Precedence 💫

So this is the part that tells you which header record or field or command line option actually wins when they all show up to the party at once. When pax is used in read or list modes, it shall determine a file attribute in exactly this sequence:

If -odelete=keyword-prefix is used, the affected attributes shall be determined from step 7., if applicable, or ignored otherwise.
If -okeyword:= is used, the affected attributes shall be ignored.
If -okeyword:=value is used, the affected attribute shall be assigned the value.
If there is a typeflag x extended header record, the affected attribute shall be assigned the <value>. When extended header records conflict, the last one given in the header shall take precedence.
If -okeyword=value is used, the affected attribute shall be assigned the value.
If there is a typeflag g global extended header record, the affected attribute shall be assigned the <value>. When global extended header records conflict, the last one given in the global header shall take precedence.
Otherwise, the attribute shall be determined from the ustar header block. (The fallback. The reliable friend. The one who always shows up. 🫶)

⏰ pax Extended Header File Times ⏰

The pax utility shall write an mtime record for each file in write or copy modes if the file's modification time cannot be represented exactly in the ustar header logical record described in ustar Interchange Format. When does that happen, you ask?

The time is out of ustar range, or
the file system of the underlying implementation supports non-integer time granularities and the time is not an integer. 🙃

All of these time records shall be formatted as a decimal representation of the time in seconds since the Epoch. If a <period> (.) decimal point character is present, the digits to the right of the point shall represent the units of a subsecond timing granularity, where the first digit is tenths of a second and each subsequent digit is a tenth of the previous digit.

In read or copy mode, the pax utility shall truncate the time of a file to the greatest value that is not greater than the input header file time.

In write or copy mode, the pax utility shall output a time exactly if it can be represented exactly as a decimal number, and otherwise shall generate only enough digits so that the same time shall be recovered if the file is extracted on a system whose underlying implementation supports the same time granularity. Efficient. Minimalist. She said "no wasted digits." 💅

📼 ustar Interchange Format 📼

A ustar archive tape or file shall contain a series of logical records. Each logical record shall be a fixed-size logical record of 512 octets (see below). And yeah, this format may be thought of as being stored on 9-track industry-standard 12.7 mm (0.5 in) magnetic tape, but other types of transportable media are not excluded — she's flexible, she contains multitudes. 🌈

Each file archived shall be represented by a header logical record that describes the file, followed by zero or more logical records that give the contents of the file. At the end of the archive file there shall be two 512-octet logical records filled with binary zeros, interpreted as an end-of-archive indicator. That's the little "we're done here, bye" at the end. ✌️

The logical records may be grouped for physical I/O operations, as described under the -bblocksize and -x ustar options. Each group of logical records may be written with a single operation equivalent to the write() function. On magnetic tape, the result of this write shall be a single tape physical block. The last physical block shall always be the full size, so logical records after the two zero logical records may contain undefined data. (Undefined! Do not read into it! 😭)

The header logical record shall be structured as shown in the following table. All lengths and offsets are in decimal.

Table 4-14: ustar Header Block

                 ┌────────────┬──────────────┬────────────────────┐
                 │ Field Name │ Octet Offset │ Length (in Octets) │
                 ├────────────┼──────────────┼────────────────────┤
                 │ name       │       0      │        100         │
                 │ mode       │     100      │          8         │
                 │ uid        │     108      │          8         │
                 │ gid        │     116      │          8         │
                 │ size       │     124      │         12         │
                 │ mtime      │     136      │         12         │
                 │ chksum     │     148      │          8         │
                 │ typeflag   │     156      │          1         │
                 │ linkname   │     157      │        100         │
                 │ magic      │     257      │          6         │
                 │ version    │     263      │          2         │
                 │ uname      │     265      │         32         │
                 │ gname      │     297      │         32         │
                 │ devmajor   │     329      │          8         │
                 │ devminor   │     337      │          8         │
                 │ prefix     │     345      │        155         │
                 └────────────┴──────────────┴────────────────────┘

All characters in the header logical record shall be represented in the coded character set of the ISO/IEC 646:1991 standard. For maximum portability between implementations, names should be selected from characters represented by the portable filename character set as octets with the most significant bit zero. If an implementation supports the use of characters outside of <slash> and the portable filename character set in names for files, users, and groups, one or more implementation-defined encodings of these characters shall be provided for interchange purposes.

However — and this is the plot twist — the pax utility shall never create filenames on the local system that cannot be accessed via the procedures described in POSIX.1-2008. If a filename is found on the medium that would create an invalid filename, it is implementation-defined whether the data from the file is stored on the file hierarchy and under what name it is stored. The pax utility may choose to ignore these files as long as it produces an error indicating that the file is being ignored.

Each field within the header logical record is contiguous; that is, there is no padding used. Each character on the archive medium shall be stored contiguously. Tight. Compact. No filler. 💖

The fields magic, uname, and gname are character strings each terminated by a NUL character. The fields name, linkname, and prefix are NUL-terminated character strings except when all characters in the array contain non-NUL characters including the last character. The version field is two octets containing the characters "00" (zero-zero). The typeflag contains a single character. All other fields are leading zero-filled octal numbers using digits from the ISO/IEC 646:1991 standard IRV. Each numeric field is terminated by one or more <space> or NUL characters.

The name and the prefix fields shall produce the pathname of the file. A new pathname shall be formed, if prefix is not an empty string (its first character is not NUL), by concatenating prefix (up to the first NUL character), a <slash> character, and name; otherwise, name is used alone. In either case, name is terminated at the first NUL character. If prefix begins with a NUL character, it shall be ignored. In this manner, pathnames of at most 256 characters can be supported.

If a pathname does not fit in the space provided, pax shall notify the user of the error, and shall not store any part of the file — header or data — on the medium. Not even a crumb. Not even a hint. 😤

The linkname field, described below, shall not use the prefix to produce a pathname. As such, a linkname is limited to 100 characters. If the name does not fit in the space provided, pax shall notify the user of the error, and shall not attempt to store the link on the medium.

The mode field provides 12 bits encoded in the ISO/IEC 646:1991 standard octal digit representation. The encoded bits shall represent the following values:

Table: ustar mode Field

  ┌───────────┬──────────────────┬─────────────────────────────────────────────────┐
  │ Bit Value │ POSIX.1-2008 Bit │                   Description                   │
  ├───────────┼──────────────────┼─────────────────────────────────────────────────┤
  │   04000   │ S_ISUID          │ Set UID on execution.                           │
  │   02000   │ S_ISGID          │ Set GID on execution.                           │
  │   01000   │ <reserved>       │ Reserved for future standardization.            │
  │   00400   │ S_IRUSR          │ Read permission for file owner class.           │
  │   00200   │ S_IWUSR          │ Write permission for file owner class.          │
  │   00100   │ S_IXUSR          │ Execute/search permission for file owner class. │
  │   00040   │ S_IRGRP          │ Read permission for file group class.           │
  │   00020   │ S_IWGRP          │ Write permission for file group class.          │
  │   00010   │ S_IXGRP          │ Execute/search permission for file group class. │
  │   00004   │ S_IROTH          │ Read permission for file other class.           │
  │   00002   │ S_IWOTH          │ Write permission for file other class.          │
  │   00001   │ S_IXOTH          │ Execute/search permission for file other class. │
  └───────────┴──────────────────┴─────────────────────────────────────────────────┘

Note 01000 is <reserved> — reserved for future standardization, so hands off, she's not yours yet. 👑

When appropriate privileges are required to set one of these mode bits, and the user restoring the files from the archive does not have appropriate privileges, then bestie… the mode bits for which they don't have appropriate privileges shall be ignored. Periodt. Some of the mode bits in the archive format aren't even mentioned anywhere else in this volume of POSIX.1‐2017, so if the implementation doesn't support those bits, it may just ignore them. 🙈

The uid and gid fields are the user and group ID of the owner and group of the file, respectively. 💖

The size field is the size of the file in octets. Okay so here's the tea ☕:

If the typeflag field is set to specify a file of type 1 (a link) or 2 (a symbolic link), the size field shall be specified as zero.
If the typeflag field is set to specify a file of type 5 (directory), the size field shall be interpreted as described under the definition of that record type.
No data logical records are stored for types 1, 2, or 5.
If the typeflag field is set to 3 (character special file), 4 (block special file), or 6 (FIFO), the meaning of the size field is unspecified by this volume of POSIX.1‐2017, and no data logical records shall be stored on the medium. Also, for type 6, the size field shall be ignored when reading.
If the typeflag field is set to any other value, the number of logical records written following the header shall be (size+511)/512, ignoring any fraction in the result of the division.

The mtime field shall be the modification time of the file at the time it was archived. It is the ISO/IEC 646:1991 standard representation of the octal value of the modification time obtained from the stat() function. 🕐

The chksum field shall be the ISO/IEC 646:1991 standard IRV representation of the octal value of the simple sum of all octets in the header logical record. Every single octet in the header shall be treated as an unsigned value. These values shall be added to an unsigned integer, initialized to zero, and the precision of that integer is not less than 17 bits. When calculating the checksum, the chksum field is treated as if it were all <space> characters. 🧮

The typeflag field specifies the type of file archived. If a particular implementation does not recognize the type, or the user does not have appropriate privileges to create that type, the file shall be extracted as if it were a regular file — but only if the file type is defined to have a meaning for the size field that could cause data logical records to be written on the medium (see the previous description for size). If conversion to a regular file occurs, the pax utility shall produce an error indicating that the conversion took place. 👀

All of the typeflag fields shall be coded in the ISO/IEC 646:1991 standard IRV:

0 — Represents a regular file. For backwards-compatibility, a typeflag value of binary zero ('\0') should be recognized as meaning a regular file when extracting files from the archive. Archives written with this version of the archive file format create regular files with a typeflag value of the ISO/IEC 646:1991 standard IRV '0'.
1 — Represents a file linked to another file, of any type, previously archived. Such files are identified by having the same device and file serial numbers, and pathnames that refer to different directory entries. All such files shall be archived as linked files. The linked-to name is specified in the linkname field with a NUL-character terminator if it is less than 100 octets in length.
2 — Represents a symbolic link. The contents of the symbolic link shall be stored in the linkname field.
3,4 — Represent character special files and block special files respectively. In this case the devmajor and devminor fields shall contain information defining the device, the format of which is unspecified by this volume of POSIX.1‐2017. Implementations may map the device specifications to their own local specification or may ignore the entry.
5 — Specifies a directory or subdirectory. On systems where disk allocation is performed on a directory basis, the size field shall contain the maximum number of octets (which may be rounded to the nearest disk block allocation unit) that the directory may hold. A size field of zero indicates no such limiting. Systems that do not support limiting in this manner should ignore the size field.
6 — Specifies a FIFO special file. Note that the archiving of a FIFO file archives the existence of this file and not its contents.
7 — Reserved to represent a file to which an implementation has associated some high-performance attribute. Implementations without such extensions should treat this file as a regular file (type 0).
A‐Z — The letters 'A' to 'Z', inclusive, are reserved for custom implementations. All other values are reserved for future versions of this standard.

It is unspecified whether files with pathnames that refer to the same directory entry are archived as linked files or as separate files. If they are archived as linked files, this means that attempting to extract both pathnames from the resulting archive will always cause an error (unless the -u option is used) because the link cannot be created. 😭

It is also unspecified whether files with the same device and file serial numbers being appended to an archive are treated as linked files to members that were in the archive before the append.

Attempts to archive a socket shall produce a diagnostic message when ustar interchange format is used, but may be allowed when pax interchange format is used. Handling of other file types is implementation-defined. 🧦

The magic field is the specification that this archive was output in this archive format. If this field contains ustar (the five characters from the ISO/IEC 646:1991 standard IRV shown followed by NUL), the uname and gname fields shall contain the ISO/IEC 646:1991 standard IRV representation of the owner and group of the file, respectively (truncated to fit, if necessary). When the file is restored by a privileged, protection-preserving version of the utility, the user and group databases shall be scanned for these names. If found, the user and group IDs contained within these files shall be used rather than the values contained within the uid and gid fields. 🔍

🦋 cpio Interchange Format 🦋

The octet-oriented cpio archive format shall be a series of entries, each comprising a header that describes the file, the name of the file, and then the contents of the file. Three-part slay. 💅

An archive may be recorded as a series of fixed-size blocks of octets. This blocking shall be used only to make physical I/O more efficient. The last group of blocks shall always be at the full size.

For the octet-oriented cpio archive format, the individual entry information shall be in the order indicated and described by the following table; see also the <cpio.h> header.

Table 4-16: Octet-Oriented cpio Archive Entry

Header fields, in order, with their lengths and how they're read:

c_magic — length 6 octets — interpreted as an octal number
c_dev — length 6 octets — interpreted as an octal number
c_ino — length 6 octets — interpreted as an octal number
c_mode — length 6 octets — interpreted as an octal number
c_uid — length 6 octets — interpreted as an octal number
c_gid — length 6 octets — interpreted as an octal number
c_nlink — length 6 octets — interpreted as an octal number
c_rdev — length 6 octets — interpreted as an octal number
c_mtime — length 11 octets — interpreted as an octal number
c_namesize — length 6 octets — interpreted as an octal number
c_filesize — length 11 octets — interpreted as an octal number

Filename field:

c_name — length c_namesize — interpreted as a pathname string

File data field:

c_filedata — length c_filesize — interpreted as data

✨ cpio Header ✨

For each file in the archive, a header as defined previously shall be written. The information in the header fields is written as streams of the ISO/IEC 646:1991 standard characters interpreted as octal numbers. The octal numbers shall be extended to the necessary length by appending the ISO/IEC 646:1991 standard IRV zeros at the most-significant-digit end of the number; the result is written to the most-significant digit of the stream of octets first. The fields shall be interpreted as follows:

c_magic — Identify the archive as being a transportable archive by containing the identifying value "070707".
c_dev, c_ino — Contains values that uniquely identify the file within the archive (that is, no files contain the same pair of c_dev and c_ino values unless they are links to the same file). The values shall be determined in an unspecified manner.
c_mode — Contains the file type and access permissions as defined in the following table.

so like, here's the whole vibe check on c_mode — every single value, no skips:

Table 4-17: Values for cpio c_mode Field

File Permissions Name — the permission bits:

C_IRUSR — 000400 — read by owner
C_IWUSR — 000200 — write by owner
C_IXUSR — 000100 — execute by owner
C_IRGRP — 000040 — read by group
C_IWGRP — 000020 — write by group
C_IXGRP — 000010 — execute by group
C_IROTH — 000004 — read by others
C_IWOTH — 000002 — write by others
C_IXOTH — 000001 — execute by others
C_ISUID — 004000 — set uid
C_ISGID — 002000 — set gid
C_ISVTX — 001000 — reserved

File Type Name — the "what even is this thing" bits:

C_ISDIR — 040000 — directory
C_ISFIFO — 010000 — FIFO
C_ISREG — 0100000 — regular file
C_ISLNK — 0120000 — symbolic link
C_ISBLK — 060000 — block special file
C_ISCHR — 020000 — character special file
C_ISSOCK — 0140000 — socket
C_ISCTG — 0110000 — reserved

Okay so on a system that conforms to this volume of POSIX.1‐2017, the ones that shall be supported are directories, FIFOs, symbolic links, and regular files. The extra values that were defined before this? They're just sitting there reserved so old systems stay happy. Implementations may support additional file types, but bestie, please don't write those into archives you're planning to ship off to other systems. That's a whole mess waiting to happen. 🫠

c_uid — has the user ID of the owner in it.
c_gid — has the group ID of the group in it.
c_nlink — holds a number that's greater than or equal to how many links in the archive point at the file. Now here's the tea: if you used the -a option to append to a cpio archive, then pax doesn't have to account for the files in the part of the archive that already existed when it's working out the c_nlink values for the newly appended part. And it also doesn't have to go back and change the c_nlink values in the existing part of the archive, even if you append more files that share the same c_dev and c_ino values. Lowkey generous of it, honestly.
c_rdev — has implementation-defined info for character or block special files.
c_mtime — has the most recent time the file was modified, as of when the archive got created.
c_namesize — has the length of the pathname, and yes that length includes the terminating NUL character. Don't forget the NUL, it counts. ✨
c_filesize — has the length, in octets, of the data section that comes after the header structure.

`cpio` Filename 🎀

The c_name field shall contain the pathname of the file. And the length of this field in octets? That's the value of c_namesize. Matching set, iconic.

If a filename shows up on the medium that would create an invalid pathname, whether the data from that file gets stored on the file hierarchy — and what name it gets stored under — is implementation-defined. So basically, your mileage may vary, and it's not the standard's problem.

All characters shall be represented in the ISO/IEC 646:1991 standard IRV. For maximum portability between implementations, names should be picked from characters represented by the portable filename character set, as octets with the most significant bit zero. If an implementation supports using characters outside the portable filename character set in names for files, users, and groups, then it shall provide one or more implementation-defined encodings of those characters so they can be interchanged. But here's the rule that never bends: the pax utility shall never create filenames on the local system that can't be accessed through the procedures described earlier in this volume of POSIX.1‐2017. Nope. Not doing it. 🙅

If a filename is found on the medium that would create an invalid filename, whether the data from the file is stored on the local file system — and under what name — is implementation-defined. And pax is allowed to just ignore those files entirely, as long as it throws an error saying it's ignoring them. Rude but valid. 💅

`cpio` File Data 🍓

Right after c_name comes c_filesize octets of data. How that data gets interpreted depends on the file. For regular files, the data shall consist of the contents of the file. For symbolic links, the data shall consist of the contents of the symbolic link. And if c_filesize is zero, then there's no data contained in c_filedata at all — nothing to see here.

Now, when you're restoring from an archive:

If the user doesn't have the appropriate privileges to create a file of the specified type, pax shall ignore that entry and write an error message to standard error.
Only regular files and symbolic links actually have data to be restored. And presuming a regular file meets whatever selection criteria the format-reading utility got handed by the user, that data shall be restored.
If a user doesn't have the appropriate privileges to set a particular mode flag, that flag shall be ignored. Some of the mode flags in the archive format aren't mentioned anywhere else in this volume of POSIX.1‐2017, so if the implementation doesn't support those flags, they may be ignored too.

`cpio` Special Entries 🦋

FIFO special files, directories, and the trailer shall all be recorded with c_filesize equal to zero. Symbolic links shall be recorded with c_filesize equal to the length of the contents of the symbolic link. For other special files? This volume of POSIX.1‐2017 leaves c_filesize unspecified. Mystery value, we don't know her.

The header for the next file entry in the archive shall be written directly after the last octet of the file entry that came before it — no gaps, no filler, just straight into it. And a header denoting the filename TRAILER!!! indicates the end of the archive. Heads up though: the contents of the octets in the last block of the archive that come after such a header are undefined. So don't go reading them and expecting anything, bestie. 😭

✨ EXIT STATUS ✨

These are the exit values that shall be returned:

0 — all files were processed successfully. 🎉 slay.
>0 — an error occurred. 💀

💀 CONSEQUENCES OF ERRORS 💀

If pax can't create a file or a link while reading an archive, or can't find a file while writing an archive, or can't preserve the user ID, group ID, or file mode when the -p option is specified, then a diagnostic message shall be written to standard error and a non-zero exit status shall be returned — but processing shall continue. It keeps going. It's got main character energy like that. And in the case where pax can't create a link to a file, by default it shall not create a second copy of the file.

Okay this next bit is important, so pay attention. 🚨 If the extraction of a file from an archive gets prematurely terminated by a signal or an error, pax may have only partially extracted the file. Or — if the -n option was not specified — it may have extracted a file with the same name as the one the user specified, but which is not the file the user actually wanted. Not the same girl, not the same file. Additionally, the file modes of extracted directories may have extra bits from the S_IRWXU mask set, plus incorrect modification and access times. Messy. Be aware.

The following sections are informative.

💅 APPLICATION USAGE 💅

Bestie, listen up, because this next bit is the "please don't hurt yourself" section and it's lowkey important.

Caution is advised when you use the -a option to append to a cpio format archive. Here's the tea: if any of the files you're appending happen to be given the same c_dev and c_ino values as a file already in the existing part of the archive, then on extraction they might get treated as links to that file. Rude, right? So it's risky to use -a with cpio format except when:

it's done on the same system that the original archive was created on,
with the same pax utility,
and in the knowledge that there's been little or no file system activity since the original archive was created that could lead to any of the files being appended getting the same c_dev and c_ino values as an unrelated file in the existing part of the archive.

Also (and this is the intentional-links case), when you're deliberately appending additional links to a file in the existing part of the archive, the c_nlink values in the modified archive can end up smaller than the number of links to the file in the archive, which may mean the links are not preserved on extraction. 😭

The -p (privileges) option was literally invented to reconcile differences between historical tar and cpio implementations. Specifically, those two utilities use -m in diametrically opposed ways, which is chaos. The -p option also gives a consistent way to extend how future file attributes can be addressed — like for enhanced security systems or high-performance files. Sounds complicated, but there are really just two modes that get used most of the time:

-p e — "Preserve everything". This is what the historical superuser — someone with all appropriate privileges — would use to preserve all aspects of the files as they're recorded in the archive. The e flag is the sum of o and p, plus other implementation-defined attributes. 👑
-p p — "Preserve" the file mode bits. This is for the user with regular privileges who wants to preserve aspects of the file other than the ownership. File times are preserved by default, but two other flags are offered to disable these and use the time of extraction instead. 🕐

The one-pathname-per-line format of standard input precludes pathnames containing <newline> characters. And yeah, those pathnames do violate the portable filename guidelines, but they may exist, and their presence may inhibit usage of pax within shell scripts. This whole problem is inherited from historical archive programs, so it's not even pax's fault. 🙈 The fix? List your filename arguments on the command line instead of on standard input. Slay.

It is almost certain that appropriate privileges are required for pax to accomplish parts of this volume of POSIX.1‐2017. Specifically:

creating files of type block special or character special,
restoring file access times unless the files are owned by the user (the -t option),
or preserving file owner, group, and mode (the -p option)

...all probably require appropriate privileges. No cap.

In read mode, implementations are permitted to overwrite files when the archive has multiple members with the same name. This may fail if permissions on the first version of the file don't permit it to be overwritten. 💀

The cpio and ustar formats can only support files up to 8589934592 bytes (8 ∗ 2^30) in size. That's the ceiling, bestie.

When archives containing binary header information are listed, the filenames printed may cause strange behavior on some terminals. 🫠

Okay so here's a fun edge case. When all of the following are true:

A file of type directory is being placed into an archive.
The ustar archive format is being used.
The pathname of the directory is less than or equal to 155 bytes long (it will fit in the prefix field in the ustar header block).
The last component of the pathname of the directory is longer than 100 bytes long (it will not fit in the name field in the ustar header block).

...some implementations of the pax utility will place the entire directory pathname in the prefix field, set the name field to an empty string, and place the directory in the archive. ✨ Other implementations will give an error under these conditions, because the name field isn't large enough to hold the last component of the directory name.

This standard allows either behavior. But — and this is the important bit — when extracting a directory from a ustar format archive, this standard requires that all implementations be able to extract a directory even if the name field contains an empty string, as long as the prefix field does not also contain an empty string. Periodt.

🌈 EXAMPLES 🌈

The following command:

pax -w -f /dev/rmt/1m .

copies the contents of the current directory to tape drive 1, medium density (assuming historical System V device naming procedures — the historical BSD device name would be /dev/rmt9). 📼

The following commands:

mkdir newdir
pax -rw olddir newdir

copy the olddir directory hierarchy to newdir. 🦋

pax -r -s ',^//*usr//*,,' -f a.pax

reads the archive a.pax, with all files rooted in /usr in the archive extracted relative to the current directory.

Using the option:

-o listopt="%M %(atime)T %(size)D %(name)s"

overrides the default output description in Standard Output and instead writes:

-rw-rw--- Jan 12 15:53 2003 1492 /usr/foo/bar

Using the options:

-o listopt='%L\t%(size)D\n%.7' \
-o listopt='(name)s\n%(atime)T\n%T'

overrides the default output description in Standard Output and instead writes:

/usr/foo/bar -> /tmp   1492
/usr/fo
Jan 12 15:53 1991
Jan 31 15:53 2003

🧚 RATIONALE 🧚

The pax utility was new for the ISO POSIX‐2:1993 standard. It represents a peaceful compromise between advocates of the historical tar and cpio utilities. A whole diplomatic summit, honestly. 🕊️

A fundamental difference between cpio and tar was in the way directories were treated. The cpio utility didn't treat directories differently from other files, and to select a directory and its contents required that each file in the hierarchy be explicitly specified. For tar, a directory matched every file in the file hierarchy it rooted.

pax offers both interfaces: by default, directories map into the file hierarchy they root. The -d option causes pax to skip any file not explicitly referenced, as cpio historically did. The tar-style behavior was chosen as the default because it was believed that this was the more common usage, and because tar is the more commonly available interface, as it was historically provided on both System V and BSD implementations. 💅

The data interchange format specification in this volume of POSIX.1‐2017 requires that processes with appropriate privileges'' shall **always** restore the ownership and permissions of extracted files *exactly* as archived. But if you look at it through the historic equivalence between superuser and appropriate privileges'', there are two problems with this requirement:

Users running as superusers may unknowingly set dangerous permissions on extracted files. 😱
It's needlessly limiting — superusers can't extract files and own them as superuser unless the archive was created by the superuser.

(It should be noted that restoration of ownerships and permissions for the superuser, by default, is historical practice in cpio, but not in tar.)

In order to avoid these two problems, the pax specification has an additional ``privilege'' mechanism: the -p option. Only a pax invocation with the privileges needed, and which has the -p option set using the e specification character, has appropriate privileges to restore full ownership and permission information. 👑

Note also that this volume of POSIX.1‐2017 requires that the file ownership and access permissions shall be set, on extraction, in the same fashion as the creat() function when provided with the mode stored in the archive. This means that the file creation mask of the user is applied to the file permissions.

Users should note that directories may be created by pax while extracting files with permissions that are different from those that existed at the time the archive was created. When extracting sensitive information into a directory hierarchy that no longer exists, users are encouraged to set their file creation mask appropriately to protect these files during extraction. 🔒

The table of contents output is written to standard output to facilitate pipeline processing. 🌊

An early proposal had hard links displaying for all pathnames. This was removed because it complicates the output of the case where -v is not specified and does not match historical cpio usage. The hard-link information is available in the -v display. iykyk.

The description of the -l option allows implementations to make hard links to symbolic links. Earlier versions of this standard did not specify any way to create a hard link to a symbolic link, but many implementations provided this capability as an extension. If there are hard links to symbolic links when an archive is created, the implementation is required to archive the hard link in the archive (unless -H or -L is specified). When in read mode and in copy mode, implementations supporting hard links to symbolic links should use them when appropriate. 🔗

The archive formats inherited from the POSIX.1‐1990 standard have certain restrictions that have been brought along from historical usage. For example, there are restrictions on the length of pathnames stored in the archive. When pax is used in copy(-rw) mode (copying directory hierarchies), the ability to use extensions from the -xpax format overcomes these restrictions. ✨

Block size lore, because you know we love it:

The default blocksize value of 5120 bytes for cpio was selected because it is one of the standard block-size values for cpio, set when the -B option is specified. (The other default block-size value for cpio is 512 bytes, and this was considered to be too small.) The default block value of 10240 bytes for tar was selected because that is the standard block-size value for BSD tar. The maximum block size of 32256 bytes (215-512 bytes) is the largest multiple of 512 bytes that fits into a signed 16-bit tape controller transfer register. 📼

There are known limitations in some historical systems that would prevent larger blocks from being accepted. Historical values were chosen to improve compatibility with historical scripts using dd or similar utilities to manipulate archives. Also, default block sizes for any file type other than character special file has been deleted from this volume of POSIX.1‐2017 as unimportant and not likely to affect the structure of the resulting archive. 💜

Okay so the implementations are allowed to mess with the block-size value based on whatever archive format is in play, or based on the device the archive is getting written to. Why? So implementations get the chance to flex on special types of devices. But bestie, do NOT do this casually — it should not be used without a great deal of consideration, because it almost certainly decreases archive portability. Like, it's giving "I did this for fun and now nobody can read my archive" energy. 💀

The intended use of the -n option was to let you extract one or more files out of the archive without processing the entire archive. The standard developers were like "ooh, significant performance advantages over historical implementations." ✨ The -n option in early proposals had three effects, not one:

It caused special characters in patterns to not be treated specially.
It caused only the first file that matched a pattern to be extracted.
It caused pax to write a diagnostic message to standard error when no file was found matching a specified pattern.

Only the second behavior is retained by this volume of POSIX.1‐2017, for many reasons:

First, it is in general not acceptable for a single option to have multiple effects.
Second, the ability to make pattern matching characters act as normal characters is useful for parts of pax other than file extraction.
Third, a finer degree of control over the special characters is useful because users may wish to normalize only a single special character in a single filename.
Fourth, given a more general escape mechanism, the previous behavior of the -n option can be easily obtained using the -s option or a sed script.
Finally, writing a diagnostic message when a pattern specified by the user is unmatched by any file is useful behavior in all cases.

In this version, the -n was removed from the copy mode synopsis of pax; it is inapplicable because there are no pattern operands specified in this mode. Not applicable, not invited, blocked. 🙈

There is another method than pax for copying subtrees in POSIX.1‐2008, described as part of the cp utility. Both methods are historical practice: cp provides a simpler, more intuitive interface, while pax offers a finer granularity of control. Each provides additional functionality to the other; in particular, pax maintains the hard-link structure of the hierarchy while cp does not. It is the intention of the standard developers that the results be similar (using appropriate option combinations in both utilities). The results are not required to be identical; there seemed insufficient gain to applications to balance the difficulty of implementations having to guarantee that the results would be exactly identical. Close enough is close enough, apparently. 🫶

A single archive may span more than one file. It is suggested that implementations provide informative messages to the user on standard error whenever the archive file is changed. Like a little "hey bestie, heads up, the file changed" moment. 👑

The -d option (do not create intermediate directories not listed in the archive) found in early proposals was originally provided as a complement to the historic -d option of cpio. It has been deleted. Gone. Not here. 💅

The -s option in early proposals specified a subset of the substitution command from the ed utility. As there was no reason for only a subset to be supported, the -s option is now compatible with the current ed specification. Since the delimiter can be any non-null character, the following usage with single <space> characters is valid:

pax -s " foo bar " ...

The -t description is worded so as to note that this may cause the access time update caused by some other activity (which occurs while the file is being read) to be overwritten. Race condition vibes, but make it documented. 😭

The default behavior of pax with regard to file modification times is the same as historical implementations of tar. It is not the historical behavior of cpio.

Because the -i option uses /dev/tty, utilities without a controlling terminal are not able to use this option. No tty, no interactive era, sorry. 🎀

The -y option, found in early proposals, has been deleted because a line containing a single <period> for the -i option has equivalent functionality. The special lines for the -i option (a single <period> and the empty line) are historical practice in cpio.

In early drafts, a -echarmap option was included to increase portability of files between systems using different coded character sets. This option was omitted because it was apparent that consensus could not be formed for it. In this version, the use of UTF‐8 should be an adequate substitute. Rip -echarmap, you would've hated to see it go. 🦋

The ISO POSIX‐2:1993 standard and ISO POSIX‐1 standard requirements for pax, however, made it very difficult to create a single archive containing files created using extended characters provided by different locales. This version adds the hdrcharset keyword to make it possible to archive files in these cases without dropping files due to translation errors.

Translating filenames and other attributes from a locale's encoding to UTF‐8 and then back again can lose information, as the resulting filename might not be byte-for-byte equivalent to the original. To avoid this problem, users can specify the -o hdrcharset=binary option, which will cause the resulting archive to use binary format for all names and attributes. Such archives are not portable among hosts that use different native encodings (e.g., EBCDIC versus ASCII-based encodings), but they will allow interchange among the vast majority of POSIX file systems in practical use. Also, the -o hdrcharset=binary option will cause pax in copy mode to behave more like other standard utilities such as cp.

If the values specified by the -o exthdr.name=value, -o globexthdr.name=value, or by $TMPDIR (if -o globexthdr.name is not specified) require a character encoding other than that described in the ISO/IEC 646:1991 standard, a path extended header record will have to be created for the file. If a hdrcharset extended header record is active for such headers, it will determine the codeset used for the value field in these extended path header records. These path extended header records always need to be created when writing an archive even if hdrcharset=binary has been specified and would contain the same (binary) data that appears in the ustar header record prefix and name fields. (In other words, an extended header path record is always required to be generated if the prefix or name fields contain non-ASCII characters even when hdrcharset=binary is also in effect for that file.) Not optional, bestie. Always generated. ✨

The -k option was added to address international concerns about the dangers involved in the character set transformations of -e (if the target character set were different from the source, the filenames might be transformed into names matching existing files) and also was made more general to protect files transferred between file systems with different {NAME_MAX} values (truncating a filename on a smaller system might also inadvertently overwrite existing files). As stated, it prevents any overwriting, even if the target file is older than the source. This version adds more granularity of options to solve this problem by introducing the -oinvalid=option—specifically the UTF‐8 and binary actions. (Note that an existing file is still subject to overwriting in this case. The -k option closes that loophole.) So like, -oinvalid handles the fine-tuning, and -k is the strict parent that says "no overwriting, periodt." 💅

Some of the file characteristics referenced in this volume of POSIX.1‐2017 might not be supported by some archive formats. For example, neither the tar nor cpio formats contain the file access time. For this reason, the e specification character has been provided, intended to cause all file characteristics specified in the archive to be retained.

It is required that extracted directories, by default, have their access and modification times and permissions set to the values specified in the archive. This has obvious problems in that the directories are almost certainly modified after being extracted and that directory permissions may not permit file creation. One possible solution is to create directories with the mode specified in the archive, as modified by the umask of the user, with sufficient permissions to allow file creation. After all files have been extracted, pax would then reset the access and modification times and permissions as necessary. It's giving "get in, do the work, restore the vibe after." 🧚

The list-mode formatting description borrows heavily from the one defined by the printf utility. However, since there is no separate operand list to get conversion arguments, the format was extended to allow specifying the name of the conversion argument as part of the conversion specification.

The T conversion specifier allows time fields to be displayed in any of the date formats. Unlike the ls utility, pax does not adjust the format when the date is less than six months in the past. This makes parsing the output more predictable. Predictability slay. 👑

The D conversion specifier handles the ability to display the major/minor or file size, as with ls, by using %-8(size)D.

The L conversion specifier handles the ls display for symbolic links.

Conversion specifiers were added to generate existing known types used for ls.

🌈 pax Interchange Format 🌈

The new POSIX data interchange format was developed primarily to satisfy international concerns that the ustar and cpio formats did not provide for file, user, and group names encoded in characters outside a subset of the ISO/IEC 646:1991 standard. The standard developers realized that this new POSIX data interchange format should be very extensible because there were other requirements they foresaw in the near future:

Support international character encodings and locale information
Support security information (ACLs, and so on)
Support future file types, such as realtime or contiguous files
Include data areas for implementation use
Support systems with words larger than 32 bits and timers with subsecond granularity

The following were not goals for this format because these are better handled by separate utilities or are inappropriate for a portable format:

Encryption
Compression
Data translation between locales and codesets
inode storage

The format chosen to support the goals is an extension of the ustar format. Of the two formats previously available, only the ustar format was selected for extensions because:

It was easier to extend in an upwards-compatible way. It offered version flags and header block type fields with room for future standardization. The cpio format, while possessing a more flexible file naming methodology, could not be extended without breaking some theoretical implementation or using a dummy filename that could be a legitimate filename.
Industry experience since the original ``tar wars'' fought in developing the ISO POSIX‐1 standard has clearly been in favor of the ustar format, which is generally the default output format selected for pax implementations on new systems.

Okay so the new format was designed with one more little goal in mind, and it's kind of the cutest one: it wanted to behave reasonably when an older tar or pax utility just... happened to read an archive it didn't understand. 🥺 Because the POSIX.1‐1990 standard said that a "format-reading utility" had to treat unrecognized typeflag values as regular files, the format could sneak all its extended info into a pseudo-regular file that sits right before each real file. Iconic loophole behavior, honestly. 💅 And there's an option given so the archive creator can set up reasonable names for these files on those older systems. The normative text also suggests reasonable file access values be used for this ustar header block — because making those header files inaccessible for convenient reading and deleting? not reasonable bestie. File permissions of 600 or 700 are suggested. ✨

The ustar typeflag field got used to carry the new format's extra functionality instead of magic or version, and here's why: the POSIX.1‐1990 standard (and, by reference, the previous version of pax) mandated what the format-reading utility does when it hits an unknown typeflag — but it was silent about those other two fields. So the typeflag was the only door left unlocked. 🚪

Early proposals for the first version of this standard contained a proposed archive format based on compatibility with the standard for tape files (ISO 1001, similar to the format used historically on many mainframes and minicomputers). And like... it was a lot. Overly complex, required considerable overhead in volume and header records. The standard developers felt it would not be acceptable to the community of POSIX developers, so it was later changed to a format more closely related to historical practice on POSIX systems. Glow-down, honestly. 💀

The prefix and name split of pathnames in ustar got replaced by the single path extended header record, for simplicity. Simple is the vibe. 🦋

The concept of a global extended header (typeflag g) was controversial. If it were applied to an archive being recorded on magnetic tape, a few unreadable blocks at the beginning of the tape could be a serious problem — a utility trying to extract as many files as possible from a damaged archive could lose a large percentage of file header information in that case. 😭 But if the archive were on a reliable medium, such as a CD‐ROM, the global extended header offers considerable potential size reductions by eliminating redundant information. So the text warns against using the global method for unreliable media, and provides a method for implanting global information in the extended header for each file, rather than in the typeflag g records. Balance, bestie. ⚖️

No facility for data translation or filtering on a per-file basis is included, because the standard developers could not invent an interface that would allow this in an efficient manner. If a filter — like encryption or compression — is to be applied to all the files, it's more efficient to apply the filter to the entire archive as a single file. The standard developers considered interfaces that would invoke a shell script for each file going into or out of the archive, but the system overhead in that approach was considered to be too high. 🙅‍♀️

One such approach would be to have filter= records that give a pathname for an executable. When the program is invoked, the file and archive would be open for standard input/output and all the header fields would be available as environment variables or command-line arguments. The standard developers did discuss such schemes, but they were omitted from POSIX.1‐2008 due to concerns about excessive overhead. Also, the program itself would need to be in the archive if it were to be used portably. 👀

There is currently no portable means of identifying the character set(s) used for a file in the file system. Therefore, pax has not been given a mechanism to generate charset records automatically. The only portable means of doing this is for the user to write the archive using the -ocharset=string command line option. This assumes that all of the files in the archive use the same encoding. The "implementation-defined" text is included to allow for a system that can identify the encodings used for each of its files. 🎀

The table of standards that accompanies the charset record description is acknowledged to be very limited. Only a limited number of character set standards is reasonable for maximal interchange. Any character set is, of course, possible by prior agreement. It was suggested that EBCDIC be listed, but it was omitted because it is not defined by a formal standard. Formal standards — and then only those with reasonably large followings — can be included here, simply as a matter of practicality. The <value>s represent names of officially registered character sets in the format required by the ISO 2375:1985 standard. 📜

The normal <comma> or <blank>-separated list rules are not followed in the case of keyword options, to allow ease of argument parsing for getopts. It's giving "convenience over consistency" and honestly? Valid. 💅

Further information on character encodings is in pax Archive Character Set Encoding/Decoding. 🔥

The standard developers have reserved keyword name space for vendor extensions. It is suggested that the format to be used is:

VENDOR.keyword

where VENDOR is the name of the vendor or organization in all uppercase letters. It is further suggested that the keyword following the <period> be named differently than any of the standard keywords, so that it could be used for future standardization, if appropriate, by omitting the VENDOR prefix. Main character energy for future-proofing. 👑

The <length> field in the extended header record was included to make it simpler to step through the records, even if a record contains an unknown format (to a particular pax) with complex interactions of special characters. It also provides a minor integrity checkpoint within the records to aid a program attempting to recover files from a damaged archive. Little field, big job. 🫶

There are no extended header versions of the devmajor and devminor fields, because the unspecified format ustar header field should be sufficient. If they are not, vendor-specific extended keywords (such as VENDOR.devmajor) should be used. DIY era. 🛠️

Device and i-number labeling of files was not adopted from cpio; files are interchanged strictly on a symbolic name basis, as in ustar. Periodt.

Just as with the ustar format descriptions, the new format makes no special arrangements for multi-volume archives. Each of the pax archive types is assumed to be inside a single POSIX file, and splitting that file over multiple volumes (diskettes, tape cartridges, and so on), processing their labels, and mounting each in the proper sequence are considered to be implementation details that cannot be described portably. Not our problem, respectfully. 🙈

The pax format is intended for interchange, not only for backup on a single (family of) systems. It is not as densely packed as might be possible for backup:

It contains information as coded characters that could be coded in binary.
It identifies extended records with name fields that could be omitted in favor of a fixed-field layout.
It translates names into a portable character set and identifies locale-related information, both of which are probably unnecessary for backup.

The requirements on restoring from an archive are slightly different from the historical wording, allowing for non-monolithic privilege to bring forward as much as possible. In particular, attributes such as "high performance file" might be broadly but not universally granted, while set-user-ID or chown() might be much more restricted. There is no implication in POSIX.1‐2008 that the security information be honored after it is restored to the file hierarchy, in spite of what might be improperly inferred by the silence on that topic. That is a topic for another standard. 😭

Links are recorded in the fashion described here because a link can be to any file type. It is desirable in general to be able to restore part of an archive selectively and restore all of those files completely. If the data is not associated with each link, it is not possible to do this. However, the data associated with a file can be large, and when selective restoration is not needed, this can be a significant burden. The archive is structured so that files that have no associated data can always be restored by the name of any link name of any link, and the user may choose whether data is recorded with each instance of a file that contains data. The format permits mixing of both types of links in a single archive; this can be done for special needs, and pax is expected to interpret such archives on input properly, despite the fact that there is no pax option that would force this mixed case on output. (When -o linkdata is used, the output must contain the duplicate data, but the implementation is free to include it or omit it when -o linkdata is not used.) 🌸

The time values are included as extended header records for those implementations needing more than the eleven octal digits allowed by the ustar format. Portable file timestamps cannot be negative. If pax encounters a file with a negative timestamp in copy or write mode, it can reject the file, substitute a non-negative timestamp, or generate a non-portable timestamp with a leading '-'. Even though some implementations can support finer file-time granularities than seconds, the normative text requires support only for seconds since the Epoch, because the ISO POSIX‐1 standard states them that way. The ustar format includes only mtime; the new format adds atime and ctime for symmetry. The atime access time restored to the file system will be affected by the -p a and -p e options. The ctime creation time (actually inode modification time) is described with appropriate privileges so that it can be ignored when writing to the file system. POSIX does not provide a portable means to change file creation time. Nothing is intended to prevent a non-portable implementation of pax from restoring the value. ⏰

The gid, size, and uid extended header records were included to allow expansion beyond the sizes specified in the regular tar header. New file system architectures are emerging that will exhaust the 12-digit size field. There are probably not many systems requiring more than 8 digits for user and group IDs, but the extended header values were included for completeness, allowing overrides for all of the decimal values in the tar header. 🧚

The standard developers intended to describe the effective results of pax with regard to file ownerships and permissions; implementations are not restricted in timing or sequencing the restoration of such, provided the results are as specified. 💖

Much of the text that talks about the extended headers is describing how they work in "write or copy modes". The copy-mode mentions are there because of the normative text, which says: "The effect of the copy shall be as if the copied files were written to an archive file and then subsequently extracted ...". And like — bestie, there is genuinely no way to test whether pax is actually generating those extended headers while it's in copy mode. But the effects still have to be as if it had. Periodt.

pax Archive Character Set Encoding/Decoding 🌈

Okay so here's the tea: there's a whole need to exchange archives of files between systems that have different native codesets. Filenames, group names, and user names all have to be preserved to the fullest extent possible when an archive gets read on the receiving platform. Now — translating the actual contents of files? Not it. That is not within the scope of the pax utility.

There's also the need to represent characters that just... don't exist on the receiving platform. And these unsupported characters cannot be automatically folded down to the local character set, because — hello — collisions. That could end up overwriting previously extracted files from the archive, or pre-existing files on the system. 💀

So for all those reasons, the codeset used to represent characters inside the extended header records of a pax archive has to be sufficiently rich to handle all the commonly used character sets. The fields that need translation include, at a minimum:

filenames
user names
group names
link pathnames

Implementations may also want to have localized extended keywords that use non-portable characters. Fancy.

The standard developers considered these options:

The archive creator specifies the well-defined name of the source codeset. The receiver must then recognize the codeset name and perform the appropriate translations to the destination codeset.
The archive creator includes within the archive the character mapping table for the source codeset used to encode extended header records. The receiver must then read the character mapping table and perform the appropriate translations to the destination codeset.
The archive creator translates the extended header records in the source codeset into a canonical form. The receiver must then perform the appropriate translations to the destination codeset.

The approach that bakes in the name of the source codeset runs into the whole codeset name registration problem, and it makes the archive useless to any pax archive decoders that don't recognize that codeset. 🙈

And because parts of an archive can get corrupted, the standard developers felt that including the character map of the source codeset was too fragile. Like, losing that one key component could make the entire archive useless. (The difference between this and the global extended header decision? That one has a workaround — duplicating extended header records on unreliable media — but that would be way too burdensome for large character set maps. No cap.)

Both of the above approaches also put an undue burden on the pax archive receiver, who'd have to handle the cross-product of all source and destination codesets. Not the vibe.

So to simplify the translation from the source codeset to the canonical form, and from the canonical form to the destination codeset, the standard developers decided that the internal representation should be a stateless encoding. A stateless encoding is one where each codepoint has the same meaning, without regard to the decoder being in a specific state. An example of a stateful encoding would be the Japanese Shift-JIS; an example of a stateless encoding would be the ISO/IEC 646:1991 standard (equivalent to 7-bit ASCII). Slay.

For all these reasons, the standard developers decided to adopt a canonical format for representing file information strings. The obvious, well-endorsed candidate is the ISO/IEC 10646‐1:2000 standard (based in part on Unicode), which can represent the characters of virtually all standardized character sets. Initially they agreed on using UCS2 (16-bit Unicode) as the internal representation — that repertoire of characters is a sufficiently rich set to represent all commonly-used codesets.

But then they found that the 16-bit Unicode representation had some problems. 😭 It forced the whole issue of standardizing byte ordering. And the 2-byte length of each character made the extended header records twice as long for strings coded entirely from historical 7-bit ASCII. So for these reasons, the standard developers chose the UTF‐8 defined in the ISO/IEC 10646‐1:2000 standard. This multi-byte representation encodes UCS2 or UCS4 characters reliably and deterministically, which eliminates the need for a canonical byte ordering. Bonus: NUL octets and other characters that could possibly confuse POSIX file systems do not appear — except to represent themselves. It was realized that certain national codesets take up more space after the encoding, because of their placement within the UCS range, but it was felt that the usefulness of the encoding of the names outweighs the disadvantage of size increase for file, user, and group names. 💅

The encoding of UTF‐8 is as follows:

UCS4 Hex Encoding  UTF-8 Binary Encoding

00000000-0000007F  0xxxxxxx
00000080-000007FF  110xxxxx 10xxxxxx
00000800-0000FFFF  1110xxxx 10xxxxxx 10xxxxxx
00010000-001FFFFF  11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
00200000-03FFFFFF  111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
04000000-7FFFFFFF  1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx

where each 'x' represents a bit value from the character being translated.

ustar Interchange Format 🦋

The description of the ustar format reflects a lot of enhancements over pre-1988 versions of the historical tar utility. The goal of these changes wasn't just to provide the functional enhancements everyone wanted — it was also to keep compatibility between new and old versions. And guess what? That compatibility has been retained. Archives written using the old archive format are compatible with the new format. Iconic behavior, honestly.

Implementors should be aware that the previous file format did not include a mechanism to archive directory type files. So for that reason, the convention of using a filename ending with <slash> was adopted to specify a directory on the archive.

The total size of the name and prefix fields have been set to meet the minimum requirements for {PATH_MAX}. If a pathname will fit within the name field, it is recommended that the pathname be stored there without the use of the prefix field. Now — the name field is known to be too small to contain {PATH_MAX} characters, but the value was not changed in this version of the archive file format, because they wanted to retain backwards-compatibility. Instead, the prefix was introduced. Also, because of the earlier version of the format, there is no way to remove the restriction on the linkname field being limited in size to just that of the name field. That's just how it is, bestie.

The size field is required to be meaningful in all implementation extensions, although it could be zero. This is required so that the data blocks can always be properly counted. 🔥

It is suggested that if device special files need to be represented that cannot be represented in the standard format, that one of the extension types (A‐Z) be used, and that the additional information for the special file be represented as data and be reflected in the size field.

Attempting to restore a special file type — where it's converted to ordinary data and conflicts with an existing filename — need not be specially detected by the utility. If run as an ordinary user, pax should not be able to overwrite the entries in, for example, /dev in any case (whether the file is converted to another type or not). If run as a privileged user, it should be able to do so, and it would be considered a bug if it did not. The same is true of ordinary data files and similarly named special files; it's impossible to anticipate the needs of the user (who could really intend to overwrite the file), so the behavior should be predictable (and thus regular) and rely on the protection system as required.

The value 7 in the typeflag field is intended to define how contiguous files can be stored in a ustar archive. POSIX.1‐2008 does not require the contiguous file extension, but it does define a standard way of archiving such files so that all conforming systems can interpret these file types in a meaningful and consistent manner. On a system that does not support extended file types, the pax utility should do the best it can with the file and go on to the next.

The file protection modes are those conventionally used by the ls utility. This is extended beyond the usage in the ISO POSIX‐2 standard to support the "shared text" or "sticky" bit. It is intended that the conformance document should not document anything beyond the existence of and support of such a mode. Further extensions are expected to these bits, particularly with overloading the set-user-ID and set-group-ID flags.

cpio Interchange Format 🎀

The reference to appropriate privileges in the cpio format refers to an error on standard output; the ustar format does not make comparable statements.

The model for this format was the historical System V cpio-c data interchange format. This model documents the portable version of the cpio format — and not the binary version, okay? It has the flexibility to transfer data of any type described within POSIX.1‐2008, yet is extensible to transfer data types specific to extensions beyond POSIX.1‐2008 (for example, contiguous files). And because it describes existing practice, there is no question of maintaining upwards-compatibility. So relaxed. ✨

cpio Header 💖

There has been some concern that the size of the c_ino field of the header is too small to handle those systems that have very large inode numbers. However — plot twist — the c_ino field in the header is used strictly as a hard-link resolution mechanism for archives. It is not necessarily the same value as the inode number of the file in the location from which that file is extracted. So breathe.

The name c_magic is based on historical usage. That's the whole story.

cpio Filename 🌸

For most historical implementations of the cpio utility, {PATH_MAX} octets can be used to describe the pathname without the addition of any other header fields (the NUL character would be included in this count). {PATH_MAX} is the minimum value for pathname size, documented as 256 bytes. However, an implementation may use c_namesize to determine the exact length of the pathname. With the current description of the <cpio.h> header, this pathname size can be as large as a number that is described in six octal digits.

Two values are documented under the c_mode field values to provide for extensibility for known file types:

0110 000 — reserved for contiguous files 🫶. The implementation may treat the rest of the info for this archive like a regular file. And if this file type is, like, undefined? The implementation may just create the file as a regular file. No drama. 💅

This whole thing is giving extensibility of the cpio format while still letting you read old archives, which is honestly so thoughtful of it 😭. Files of an unknown type may be read as ``regular files'' on some implementations. And on a system that doesn't support extended file types, the pax utility should do the best it can with the file and go on to the next. Queen behavior fr fr 👑.

🔮 FUTURE DIRECTIONS

None. ✨ That's it. Nothing on the horizon. The future is a blank canvas bestie 🎀.

👀 SEE ALSO

Chapter 2, Shell Command Language, cp, ed, getopts, ls, printf
The Base Definitions volume of POSIX.1‐2017, Section 3.169, File Mode Bits, Chapter 5, File Format Notation, Chapter 8, Environment Variables, Section 12.2, Utility Syntax Guidelines, <cpio.h>, <tar.h>
The System Interfaces volume of POSIX.1‐2017, chown(), creat(), fstatat(), mkdir(), mkfifo(), utime(), write()

📜 COPYRIGHT

Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1-2017, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 7, 2018 Edition, Copyright (C) 2018 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group.

In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. Like, it's the main character and everyone else is just a rumor 🫡.

The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .

Any typographical or formatting errors that appear in this page are most likely to have been introduced during the conversion of the source files to man page format. To report such errors, see https://www.kernel.org/doc/man-pages/reporting_bugs.html .

And that's a wrap on the whole pax page bestie, we literally ate and left zero crumbs 💖🦋 The version on the tin is IEEE/The Open Group, dated 2017, and the page answers to the name PAX(1P) — screenshot it, cite it, slay with it 💅✨🌸

📦 where this page comes from

alpine 3.22: man-pages-posix 2017a-r0 · custom
alpine 3.23: man-pages-posix 2017a-r0 · custom
alpine 3.24: man-pages-posix 2017a-r0 · custom
arch rolling: man-pages 6.19-1
arch rolling: man-pages 6.20-1 · BSD-2-Clause AND BSD-3-Clause AND BSD-4.3TAHOE AND BSD-4-Clause-UC AND GPL-1.0-or-later AND GPL-2.0-only AND GPL-2.0-or-later AND GPL-3.0-or-later AND LGPL-3.0-linking-exception AND LGPL-3.0-or-later AND LicenseRef-Public-Domain AND LicenseRef-UltraPermissive AND Linux-man-pages-1-para AND Linux-man-pages-copyleft AND Linux-man-pages-copyleft-2-para AND Linux-man-pages-copyleft-var AND MIT AND Spencer-94
nixpkgs unstable: man-pages-posix 2017a
opensuse-leap 16.0: man-pages-posix 2017a-160000.4.1 · SUSE-IEEE
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-10-18: a language model's rewrite of the original, under its terms