DIFF(1P)

๐Ÿ’… DIFF(1P) โ€” the POSIX Programmer's Manual, but make it make sense

okay bestie so this is the diff page from the POSIX Programmer's Manual. manual name's right there: DIFF(1P), POSIX Programmer's Manual, DIFF(1P). that's the whole header vibe. nowโ€”

๐ŸŒธ PROLOG ๐ŸŒธ

real talk, this page is part of the POSIX Programmer's Manual. the Linux implementation of this interface may differ โ€” so go check the corresponding Linux manual page for the actual deets on Linux behavior. or, plot twist, the interface might not even be implemented on Linux at all. ๐Ÿ’€


โœจ NAME โœจ

diff โ€” compare two files ๐Ÿซถ


๐ŸŽ€ SYNOPSIS ๐ŸŽ€

diff [-c|-e|-f|-u|-C n|-U n] [-br] file1 file2

๐Ÿ’– DESCRIPTION ๐Ÿ’–

so diff compares the contents of file1 and file2, and it writes to standard output a list of the changes you'd need to make to turn file1 into file2. that list should be minimal โ€” it's giving efficiency, it's giving no wasted effort. and if the files are literally identical? no output at all. nothing. silence. ๐Ÿคซ no notes, periodt.


๐Ÿ”ฅ OPTIONS ๐Ÿ”ฅ

diff shall conform to the Base Definitions volume of POSIX.1โ€2017, Section 12.2, Utility Syntax Guidelines. (iykyk)

the following options shall be supported:

  • -b โ€” any amount of white space at the end of a line gets treated as a single <newline>. so basically the white-space characters chilling right before the <newline> are ignored. and other strings of white-space characters โ€” not counting <newline> characters โ€” compare as equal. โœจ
  • -c โ€” produce output in a form that gives you three lines of copied context.
  • -C n โ€” same vibe, but you get n lines of copied context (where n shall be interpreted as a positive decimal integer).
  • -e โ€” produce output in a form suitable as input for the ed utility, which can then be used to convert file1 into file2. it's giving pipeline energy.
  • -f โ€” produce output in an alternative form, similar in format to -e, but not intended to be suitable as input for the ed utility, and in the opposite order. so like, -e's chaotic cousin. ๐Ÿ™ˆ
  • -r โ€” apply diff recursively to files and directories of the same name when file1 and file2 are both directories.
okay so important: the diff utility shall detect infinite loops โ€” that is, entering a previously visited directory that is an ancestor of the last file encountered. when it detects an infinite loop, diff shall write a diagnostic message to standard error and shall either recover its position in the hierarchy or terminate. so it either gets itself together or it's done. ๐Ÿ’€
  • -u โ€” produce output in a form that gives you three lines of unified context.
  • -U n โ€” same but with n lines of unified context (where n shall be interpreted as a non-negative decimal integer). yes, non-negative here, not positive. different rules, bestie, don't mix them up. ๐Ÿ‘‘

๐ŸŒˆ OPERANDS ๐ŸŒˆ

the following operands shall be supported:

  • file1, file2 โ€” a pathname of a file to be compared. if either the file1 or file2 operand is -, the standard input shall be used in its place. ๐ŸŽ€

if both file1 and file2 are directories, diff shall not compare block special files, character special files, or FIFO special files to any files, and shall not compare regular files to directories. further details are as specified in Diff Directory Comparison Format. the behavior of diff on other file types is implementation-defined when found in directories. so like, unknown territory, nobody's promising anything. ๐Ÿ™ˆ

if only one of file1 and file2 is a directory, diff shall be applied to the non-directory file and the file contained in the directory file with a filename that is the same as the last component of the non-directory file. matchy matchy. โœจ


๐Ÿฆ‹ STDIN ๐Ÿฆ‹

standard input shall be used only if one of the file1 or file2 operands references standard input. see the INPUT FILES section. (go look, it's right there ๐Ÿ‘‡)


๐Ÿ“ INPUT FILES ๐Ÿ“

the input files may be of any type. literally any. no gatekeeping. ๐Ÿ’…


๐Ÿ’œ ENVIRONMENT VARIABLES ๐Ÿ’œ

the following environment variables shall affect the execution of diff:

  • LANG โ€” provide a default value for the internationalization variables that are unset or null. (see the Base Definitions volume of POSIX.1โ€2017, Section 8.2, Internationalization Variables for the precedence of internationalization variables used to determine the values of locale categories.)
  • LC_ALL โ€” if set to a non-empty string value, override the values of all the other internationalization variables. it's the main character, it overrules everyone. ๐Ÿ‘‘
  • LC_CTYPE โ€” determine the locale for the interpretation of sequences of bytes of text data as characters (for example, single-byte as opposed to multi-byte characters in arguments and input files).
  • LC_MESSAGES โ€” determine the locale that should be used to affect the format and contents of diagnostic messages written to standard error and informative messages written to standard output.
  • LC_TIME โ€” determine the locale for affecting the format of file timestamps written with the -C and -c options. โฐ
  • NLSPATH โ€” determine the location of message catalogs for the processing of LC_MESSAGES.
  • TZ โ€” determine the timezone used for calculating file timestamps written with a context format. if TZ is unset or null, an unspecified default timezone shall be used. (unspecified. we simply do not know. ๐Ÿ’€)

โญ ASYNCHRONOUS EVENTS โญ

default. that's it. that's the section. no drama, we love a low-maintenance queen. โœจ


โœจ STDOUT โœจ

okay bestie, this is the part where we talk about what actually comes OUT of diff, and there's a LOT of formats, so buckle up ๐Ÿ’–


๐ŸŒธ Diff Directory Comparison Format ๐ŸŒธ

If both file1 and file2 are directories, you get these formats, periodt.

In the POSIX locale, every file that lives in only ONE of the two directories gets reported like this:

"Only in %s: %s\n", <directory pathname>, <filename>

And subdirectories that both directories share? In the POSIX locale, they may be reported like this:

"Common subdirectories: %s and %s\n", <directory1 pathname>,
    <directory2 pathname>

Now, for each file the two directories have in common โ€” if the two files are not to be compared (i.e. they have the same device ID and file serial number, or they're both block special files pointing at the same device, or they're both character special files pointing at the same device), then in the POSIX locale the output format is unspecified ๐Ÿ™ˆ. Otherwise, in the POSIX locale, some unspecified format gets used that contains the pathnames of the two files.

For each file common to both directories, if the files ARE compared and they're identical โ€” no output, nothing, silence, slay. But if the two files differ, you get:

"diff %s %s %s\n", <diff_options>, <filename1>, <filename2>

where <diff_options> are the options exactly as you specified them on the command line. ๐Ÿ’…

Heads up: all directory pathnames listed in this section are relative to the original command line arguments. And all the other file names listed here are filenames (pathname components). Don't mix them up bestie ๐Ÿ˜ญ

๐ŸŒˆ Diff Binary Output Format ๐ŸŒˆ

In the POSIX locale, if one or both of the files being compared are not text files, it's implementation-defined whether diff uses the binary file output format or the other formats described below. The binary file output format contains the pathnames of the two files being compared and the string differ.

If both files being compared ARE text files, then depending on the options you picked, one of the following formats is used to write out the differences. ๐Ÿ‘‘


๐ŸŽ€ Diff Default Output Format ๐ŸŽ€

The default (so, no -e, no -f, no -c, no -C, no -u, no -U) diff output contains lines of these forms:

"%da%d\n", <num1>, <num2>

"%da%d,%d\n", <num1>, <num2>, <num3>

"%dd%d\n", <num1>, <num2>

"%d,%dd%d\n", <num1>, <num2>, <num3>

"%dc%d\n", <num1>, <num2>

"%d,%dc%d\n", <num1>, <num2>, <num3>

"%dc%d,%d\n", <num1>, <num2>, <num3>

"%d,%dc%d,%d\n", <num1>, <num2>, <num3>, <num4>

These lines literally look like ed subcommands for converting file1 into file2. The line numbers before the action letters pertain to file1; the ones after pertain to file2. So if you swap a for d and read the line backwards, you can also figure out how to convert file2 into file1 ๐Ÿ”„. And just like in ed, identical pairs (where num1 = num2) get abbreviated as a single number.

After each of those lines, diff writes to standard output all the lines affected in the first file using:

"< %s", <line>

and all the lines affected in the second file using:

"> %s", <line>

If there are lines affected in BOTH file1 and file2 (like with the c subcommand), the changes get separated with a line made of three <hyphen-minus> characters:

"---\n"

๐Ÿ’œ Diff -e Output Format ๐Ÿ’œ

With the -e option, a script gets produced that โ€” when you feed it to ed, along with an appended w (write) command โ€” converts file1 into file2. Only these ed commands get used in the script: a (append), c (change), d (delete), i (insert), and s (substitute). Text lines get output exactly as they appear in the file โ€” except for lines consisting of the single character <period> ('.'). ๐ŸŒธ


๐Ÿฆ‹ Diff -f Output Format ๐Ÿฆ‹

With the -f option, you get an alternative format of script. It's similar to what -e produces, but with these differences:

  1. It's expressed in reverse sequence. The output of -e orders changes from the end of the file to the beginning; -f goes from beginning to end.
  2. The command form <lines> <command-letter> used by -e is flipped. For example, 10c with -e would be c10 with -f.
  3. Ranges of line numbers are separated with a <space> instead of a <comma>.

๐Ÿ’… Diff -c or -C Output Format ๐Ÿ’…

With -c or -C, the output format consists of the affected lines plus surrounding lines of context. The affected lines show which ones need to be deleted or changed in file1, and which ones were added from file2. With -c, three lines of context (if available) get written before and after the affected lines. With -C, you get to specify how many lines of context are written. Here's the exact format:

The name and last modification time of each file get output like this:

"*** %s %s\n", file1, <file1 timestamp>
"--- %s %s\n", file2, <file2 timestamp>

Each <file> field is the pathname of the corresponding file being compared. The pathname written for standard input is unspecified. ๐Ÿ™ˆ

In the POSIX locale, each <timestamp> field is equivalent to the output of:

date "+%a %b %e %T %Y"

minus the trailing <newline>, executed at the time of last modification of the corresponding file (or the current time, if the file is standard input).

Then, these output formats get applied for every set of changes. ๐Ÿ’–

First, a line gets written like this:

"***************\n"

Next, the range of lines in file1 gets written like this, if the range contains two or more lines:

"*** %d,%d ****\n", <beginning line number>, <ending line number>

and like this otherwise:

"*** %d ****\n", <ending line number>
The ending line number of an empty range is the number of the preceding line, or 0 if the range is at the start of the file. ๐Ÿซถ

Next, the affected lines along with the lines of context (the unaffected lines) get written. Unaffected lines:

"  %s", <unaffected_line>

Deleted lines:

"- %s", <deleted_line>

Changed lines:

"! %s", <changed_line>

Next, the range of lines in file2 gets written like this, if the range contains two or more lines:

"--- %d,%d ----\n", <beginning line number>, <ending line number>

and like this otherwise:

"--- %d ----\n", <ending line number>

Then, lines of context and changed lines get written just like in the previous formats. And lines added from file2 get written like this:

"+ %s", <added_line>

๐Ÿ‘‘ Diff -u or -U Output Format ๐Ÿ‘‘

The -u or -U options behave like -c or -C, except that the context lines aren't repeated โ€” instead, the context, deleted, and added lines get shown together, interleaved. So much cleaner, lowkey iconic. The exact format:

The name and last modification time of each file get output like this:

"--- %s\t%s%s %s\n", file1, <file1 timestamp>, <file1 frac>, <file1 zone>
"+++ %s\t%s%s %s\n", file2, <file2 timestamp>, <file2 frac>, <file2 zone>

Each <file> field is the pathname of the corresponding file being compared, or the single character '-' if standard input is being compared. BUT โ€” if the pathname contains a <tab> or a <newline>, or if it doesn't consist entirely of characters from the portable character set, the behavior is implementation-defined. ๐Ÿ˜ญ

Each <timestamp> field is equivalent to the output of:

date '+%Y-%m-%d %H:%M:%S'

minus the trailing <newline>, executed at the time of last modification of the corresponding file (or the current time, if the file is standard input).

Each <frac> field is either empty, or a decimal point followed by at least one decimal digit, showing the fractional-seconds part (if any) of the file timestamp. The number of fractional digits is at least the number needed to represent the file's timestamp without losing information. โœจ

Each <zone> field is of the form shhmm, where shh is a signed two-digit decimal number in the range -24 through +25, and mm is an unsigned two-digit decimal number in the range 00 through 59. It represents the timezone of the timestamp as the number of hours (hh) and minutes (mm) east (+) or west (-) of UTC for that timestamp. If the hours and minutes are both zero, the sign is '+'. However โ€” if the timezone isn't an integral number of minutes away from UTC, the <zone> field is implementation-defined. ๐Ÿ™ˆ

Then, these output formats get applied for every set of changes.

First, the range of lines in each file gets written like this:

"@@ -%s +%s @@", <file1 range>, <file2 range>

Each <range> field is of the form:

"%1d", <beginning line number>

or:

"%1d,1", <beginning line number>

if the range contains exactly one line, and:

"%1d,%1d", <beginning line number>, <number of lines>

otherwise. If a range is empty, its beginning line number is the number of the line just before the range, or 0 if the empty range starts the file.

Next, the affected lines along with the lines of context get written. Each non-empty unaffected line:

" %s", <unaffected_line>

where the contents of the unaffected line are taken from file1. It's implementation-defined whether an empty unaffected line gets written as an empty line or as a line containing a single <space> character. This line also represents the same line of file2, even though file2's line may have different contents because of the -b. ๐Ÿ˜ณ

Deleted lines:

"-%s", <deleted_line>

Added lines:

"+%s", <added_line>

The order of the lines written is the same as the order in the corresponding file. And a deleted line is never written immediately after an added line. ๐Ÿšซ

If -U n is specified, the output contains no more than 2n consecutive unaffected lines; and if the output contains an affected line and that line is adjacent to up to n consecutive unaffected lines in the corresponding file, the output contains those unaffected lines. And -u acts like -U3. ๐Ÿ’–


๐Ÿ’€ STDERR ๐Ÿ’€

Standard error is used only for diagnostic messages. That's it, that's the whole vibe.


๐ŸŒˆ OUTPUT FILES ๐ŸŒˆ

None. Nada. Nothing. ๐ŸŽ€


๐Ÿฆ‹ EXTENDED DESCRIPTION ๐Ÿฆ‹

None either bestie, we love a low-maintenance section โœจ


โญ EXIT STATUS โญ

Here's what you get back:

  • 0 โ€” no differences were found. ๐Ÿฅณ
  • 1 โ€” differences were found. ๐Ÿ‘€
  • >1 โ€” an error occurred. ๐Ÿ’€

๐Ÿ’œ CONSEQUENCES OF ERRORS ๐Ÿ’œ

Default.


The following sections are informative.

๐ŸŒธ APPLICATION USAGE ๐ŸŒธ

okay so here's the tea, bestie โ€” if lines at the end of a file get changed and other lines get added, diff output might show that as a delete and an add, or as a change, or as a change and an add. diff is literally not expected to know which one actually happened ๐Ÿ’€, and users should not care about the difference in output, as long as it clearly shows the differences between the files. that's the whole vibe check: did it show the diff? yes? we're good.


โœจ EXAMPLES โœจ

scenario time!! Say dir1 is a directory containing a directory named x, dir2 is a directory containing a directory named x, dir1/x and dir2/x both contain files named date.out, and dir2/x contains a file named y. Then this command:

diff -r dir1 dir2

could produce output similar to:

Common subdirectories: dir1/x and dir2/x
Only in dir2/x: y
diff -r dir1/x/date.out dir2/x/date.out
1c1
< Mon Jul  2 13:12:16 PDT 1990
---
> Tue Jun 19 21:41:39 PDT 1990

see how it just walks the whole tree and reports as it goes? iconic behavior honestly ๐Ÿซถ


๐Ÿง  RATIONALE ๐Ÿง 

welcome to the lore section, this is where the standard developers explain their choices and honestly? they ate.

The -h option was omitted because it was insufficiently specified and does not add to applications portability. so long, -h ๐Ÿ’…

Historical implementations employ algorithms that do not always produce a minimum list of differences; the current language about making every effort is the best this volume of POSIX.1โ€2017 can do, as there is no metric that could be employed to judge the quality of implementations against any and all file contents. The statement ``This list should be minimal'' clearly implies that implementations are not expected to provide the following output when comparing two 100-line files that differ in only one character on a single line:

1,100c1,100
all 100 lines from file1 preceded with "< "
---
all 100 lines from file2 preceded with "> "

likeโ€ฆ that would be so unserious. one char changed and you're dumping all 100 lines?? no bestie ๐Ÿ˜ญ

The Only in'' messages required when the `-r` option is specified are not used by most historical implementations if the `-e` option is also specified. It is required here because it provides useful information that must be provided to update a target directory hierarchy to match a source hierarchy. The Common subdirectories'' messages are written by System V and 4.3 BSD when the -r option is specified. They are allowed here but are not required because they are reporting on something that is the same, not reporting a difference, and are not needed to update a target hierarchy. ngl that's a really clean distinction: "reporting a diff" vs "reporting a sameness".

The -c option, which writes output in a format using lines of context, has been included. The format is useful for a variety of reasons, among them being much improved readability and the ability to understand difference changes when the target file has line numbers that differ from another similar, but slightly different, copy. The patch utility is most valuable when working with difference listings using a context format. The BSD version of -c takes an optional argument specifying the amount of context. Rather than overloading -c and breaking the Utility Syntax Guidelines for diff, the standard developers decided to add a separate option for specifying a context diff with a specified amount of context (-C). Also, the format for context diffs was extended slightly in 4.3 BSD to allow multiple changes that are within context lines from each other to be merged together. The output format contains an additional four <asterisk> characters after the range of affected lines in the first filename. This was to provide a flag for old programs (like old versions of patch) that only understand the old context format. The version of context described here does not require that multiple changes within context lines be merged, but it does not prohibit it either. The extension is upwards-compatible, so any vendors that wish to retain the old version of diff can do so by adding the extra four <asterisk> characters (that is, utilities that currently use diff and understand the new merged format will also understand the old unmerged format, but not vice versa). backwards compatible but not forwards, iykyk โœจ

The -u and -U options of GNU diff have been included. Their output format, designed by Wayne Davison, takes up less space than -c and -C format, and in many cases is easier to read. The format's timestamps do not vary by locale, so LC_TIME does not affect it. The format's line numbers are rendered with the %1d format, not %d, because the file format notation rules would allow extra <blank> characters to appear around the numbers. shoutout to Wayne, format king ๐Ÿ‘‘

The substitute command was added as an additional format for the -e option. This was added to provide implementations with a way to fix the classic ``dot alone on a line'' bug present in many versions of diff. Since many implementations have fixed this bug, the standard developers decided not to standardize broken behavior, but rather to provide the necessary tool for fixing the bug. One way to fix this bug is to output two periods whenever a lone period is needed, then terminate the append command with a period, and then use the substitute command to convert the two periods into one period. period-ception, we love a workaround ๐Ÿ’…

The BSD-derived -r option was added to provide a mechanism for using diff to compare two file system trees. This behavior is useful, is standard practice on all BSD-derived systems, and is not easily reproducible with the find utility.

The requirement that diff not compare files in some circumstances, even though they have the same name, is based on the actual output of historical implementations. The specified behavior precludes the problems arising from running into FIFOs and other files that would cause diff to hang waiting for input with no indication to the user that diff was hung. An earlier version of this standard specified the output format more precisely, but in practice this requirement was widely ignored and the benefit of standardization seemed small, so it is now unspecified. In most common usage, diff -r should indicate differences in the file hierarchies, not the difference of contents of devices pointed to by the hierarchies. โš ๏ธ important: nobody wants a hung diff with zero explanation, that's a bad time fr

Many early implementations of diff require seekable files. Since the System Interfaces volume of POSIX.1โ€2017 supports named pipes, the standard developers decided that such a restriction was unreasonable. Note also that the allowed filename - almost always refers to a pipe. the humble dash doing the most, as always ๐Ÿซถ

No directory search order is specified for diff. The historical ordering is, in fact, not optimal, in that it prints out all of the differences at the current level, including the statements about all common subdirectories before recursing into those subdirectories.

The message:

"diff %s %s %s\n", <diff_options>, <filename1>, <filename2>

does not vary by locale because it is the representation of a command, not an English sentence. it's a command, not a sentence, periodt ๐Ÿ’…


๐Ÿ”ฎ FUTURE DIRECTIONS ๐Ÿ”ฎ

None. empty. nada. the future is a blank page bestie ๐Ÿฆ‹


๐Ÿ‘€ SEE ALSO ๐Ÿ‘€

  • cmp
  • comm
  • ed
  • find
  • The Base Definitions volume of POSIX.1โ€2017, Chapter 8, Environment Variables, Section 12.2, Utility Syntax Guidelines

Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1-2017, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 7, 2018 Edition, Copyright (C) 2018 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .

Any typographical or formatting errors that appear in this page are most likely to have been introduced during the conversion of the source files to man page format. To report such errors, see https://www.kernel.org/doc/man-pages/reporting_bugs.html .


and that's the whole page, bestie ๐ŸŽ€ you just read the entire DIFF(1P) man page and honestly? slay. this little beauty is brought to you by IEEE/The Open Group, dated 2017, and it's filed under DIFF(1P) โ€” go forth and diff responsibly, no cap ๐ŸŒˆโœจ๐Ÿ’–

๐Ÿ“ฆ where this page comes from