MAGIC(3)

📦 version 93vu, in opensuse-leap 16.0

💅 MAGIC(3) — the magic file interface, bestie edition

okay so this is from the Library Functions Manual and honestly? it's giving "ancient Unix lore but make it fetch". sit down, grab a strawberry drink 🍓, we're learning the magic file interface — no cap, it's iconic.

✨ NAME ✨

magic — the magic file interface. that's it, that's the vibe. 🎀

🦋 SYNOPSIS 🦋

#include <magic.h>

Magic_t
{
     unsigned long  flags;
};

Magic_t*  magicopen(unsigned long flags);
void      magicclose(Magic_t* magic);

int       magicload(Magic_t* magic, const char* path, unsigned long flags);
int       magiclist(Magic_t* magic, Sfio_t* sp);

char*     magictype(Magic_t* magic, const char* path, struct stat* st);

literally copy this exactly, i'm begging 🫶

🌸 DESCRIPTION 🌸

these routines are your main character entrance into the file(1) command's magic file. so slay. magicopen hands you back a magic session handle, and like… you pass that handle to every other routine in here. it's the group chat invite link, basically. 💌

flags can be any of these:

  • MAGIC_MIME — return the MIME type string instead of the magic file description. you want the vibe label, not the essay.
  • MAGIC_PHYSICAL — don't follow symbolic links. we're staying in our lane, no chasing.
  • MAGIC_STAT — the stat structure st you pass to magictype will contain valid stat(2) information. see magictype below, i got you.
  • MAGIC_VERBOSE — turns on verbose error messages. so when it goes wrong you get the full story 😭

magicclose closes the magic session. bye bestie, thanks for coming 👋

magicload loads the magic file named by path into the magic session. flags are the same as with magicopen. and here's the tea — more than one magic file can be loaded into a session, and the files are searched in load order. so order matters, periodt. if path is 0 then the default magic file gets loaded.

magiclist lists the magic file contents on the sfio(3) stream sp. it's the debugging magic entries tool, aka the "what is actually in here" button 🔍

magictype returns the type string for path with optional stat(2) information st. here's the whole flowchart, stay with me:

  • if st == 0 then magictype calls stat on a private stat buffer.
  • else if magicopen was called with the MAGIC_STAT flag then st is assumed to contain valid stat information.
  • otherwise magictype calls stat on st.

magictype always returns a non-null string. always. if errors are encountered on path then the return value will contain information on those errors, e.g. cannot stat. so even the failure is informative, we love a communicative king 👑

🧚 FORMAT 🧚

the magic file format is a backwards compatible extension of an ancient System V file implementation. yes, ancient. we respect the elders. 🙏

but with the extended format you can write a single magic file that works on all platforms, which is highkey a serve. most of the net magic files floating around work with magic, but they usually double up on le and be entries that are automatically handled by magic. so like, redundant. do less. 💅

a magic file entry describes a procedure for determining a single file type based on the file pathname, stat(2) information, and the file data.

an entry is a sequence of lines, each line being a record of space separated fields. the general record format is:

[op]offset type [mask]expression description [mimetype]

# in the first column introduces a comment. the first record in an entry contains no op; the remaining records for an entry contain an op.

integer constants are as in C: 0x* or 0X* for hexadecimal, 0* for octal and decimal otherwise. cute and classic.

the op field

the op field may be one of:

  • + — the previous records must match but the current record is optional. > is an old-style synonym for +. (legacy icon behavior, we stan both)
  • & — the previous and current records must match.
  • { — starts a nesting block that is terminated by }. a nesting block pushes a new context for the + and & ops. the { and } records have no other fields.
  • id{ — a function declaration and call for the single character identifier id. the function return is a nesting block end record }. function may be redefined. functions have no arguments or return value.
  • id() — a call to the function id.

the offset field

the offset field is either the offset into the data upon which the current entry operates, or a file metadata identifier.

offsets are either integer constants or offset expressions. an offset expression is contained in (...) and is a combination of integral arithmetic operators and the @ indirection operator. indirections take the form @integer where integer is the data offset for the indirection value.

the size of the indirection value is taken either from one of the suffixes B(byte,1char), H(short,2chars), L(long,4chars), pr Q(quead,8chars), or from the type field. (yes, that's literally how it's written, i don't make the rules 🫠)

valid file metadata identifiers are:

  • atime — the string representation of stat.st_atime.
  • blocks — stat.st_blocks.
  • ctime — the string representation of stat.st_ctime.
  • fstype — the string representation of stat.st_fstype.
  • gid — the string representation of stat.st_gid.
  • The — the stat.st_mode file mode bits in modecanon(3) canonical representation (i.e., the good old octal values).
  • mtime — the string representation of stat.st_mtime.
  • nlink — stat.st_nlink.
  • size — stat.st_size.
  • name — the file path name sans directory.
  • uid — the string representation of stat.st_uid.

the type field

the type field specifies the type of the data at offset.

integral types may be prefixed by le or be for specifying exact little-endian or big-endian representation, but the internal algorithm automatically loops through the standard representations to find integral matches, so representation prefixes are rarely used. however — this looping may cause some magic entry conflicts; use the le or be prefix in these cases. only one representation is used for all the records in an entry. noted, bestie. 📝

valid types are:

  • byte — a 1 byte integer.
  • short — a 2 byte integer.
  • long — a 4 byte integer.
  • quad — an 8 byte integer. tests on this type may fail is the local compiler does not support an 8 byte integral type and the corresponding value overflows 4 bytes.
  • date — the data at offset is interpreted as a 4 byte seconds-since-the-epoch date and converted to a string.
  • edit — the expression field is an ed(1) style substitution expression del old del new del [ flags ] where the substituted value is made available to the description field %s format. in addition to the flags supported by ed(3) are l that converts the substituted value to lower case and u that converts the substituted value to upper case. if old does not match the string data at offset then the entry record fails. 💀
  • match — expression field is a strmatch(3) pattern that is matched against the string data at offset.
  • string — the expression field is a string that is compared with the string data at offset.

the optional mask field

the optional mask field takes the form &number where number is anded with the integral value at offset before the expression is applied.

the expression field

the contents of the expression field depends on the type. string type expressions are described in the type field entries above. * means any value and applies to all types.

integral type expressions take the form [operator] operand where operand is compared with the data value at offset using operator. operator may be one of <, <=, ==, >= or >. operator defaults to == if omitted.

operand may be an integral constant or one of the following builtin function calls:

  • magic() — a recursive call to the magic algorithm starting with the data at offset. it's giving recursion, it's giving main character energy 🔥
  • loop(function,offset,increment) — call function starting at offset and increment offset by increment after each iteration. iteration continues until the description text does not change.

the description field

the description field is the most important because it is this field that is presented to the outside world. 👑 when constructing description fields one must be very careful to follow the style layed out in the magic file, lest yet another layer of inconsistency creep into the system. (the drama of it all 😭)

the description for each matching record in an entry are concatenated to form the complete magic type. if the previous matching description in the current entry does not end with space and the current description is not empty and does not start with comma, dot or backspace then a space is placed between the descriptions (most optional descriptions start with comma).

the data value at offset can be referenced in the description using %s for the string types and %ld or %lu for the integral types. format specifiers, we love to see them ✨

the mimetype field

the mimetype field specifies the MIME type, usually in the form a/b.

🎀 FILES 🎀

../lib/file/magic located on $PATH

🌈 EXAMPLES 🌈

okay these are literally the best part, look at these little guys:

0    long      0x020c0108     hp s200 executable, pure
o{
+36  long      >0        , not stripped
+4   short          >0        , version %ld
}
0    long      0x020c0107     hp s200 executable
o()
0    long      0x020c010b     hp s200 executable, demand-load
o()

the function o(), shared by 3 entries, determines if the executable is stripped and also extracts the version number. shared function, no notes, obsessed 🫶

0    long      0407      bsd 386 executable
&mode     long      &0111!=0
+16  long      >0        , not stripped

this entry requires that the file also has execute permission. permission check = vibe check ✅

🦋 SEE ALSO 🦋

file(1), mime(4), tw(1), modecanon(3)

and that's the whole page, bestie!! 💖 it's the MAGIC(3) page from the Library Functions Manual — yeah, that's the source right there, no date on this one so don't come for me — go forth and read your magic files with confidence. periodt. 💅✨🎀

📦 where this page comes from