unidesc(1)

💖 unidesc(1) — the manual, but make it a group chat 💖

okay bestie so you found unidesc, the General Commands Manual girlie, and she is literally here to tell you what's inside your Unicode text file. no cap. let's get into it 🦋

---

✨ NAME ✨

unidesc — Describe the contents of a Unicode text file

---

🌸 SYNOPSIS 🌸

unidesc ([option flags]) (<file name>)

and if you don't hand her an input file name? she just reads from the standard input. low maintenance queen 💅

---

👑 DESCRIPTION 👑

unidesc describes the content of a Unicode text file by reporting the character ranges that different portions of the text belong to. and the ranges she reports? both the official Unicode ranges and the constructed language ranges inside the Private Use Areas registered with the Conscript Unicode Registry (http://www.evertype.com/standards/csur/). iykyk 🧚

for each range of characters, unidesc prints:

• the character or byte offset of the beginning of the range
• the character or byte offset of the end of the range
• the name of the range

and bestie, the offsets start from 0. periodt.

---

okay so here's the thing about neutral characters. since the ASCII digits, punctuation, and whitespace characters get used constantly by other writing systems, by default these characters are treated as neutral — meaning they don't belong exclusively to any particular character range. instead, they're treated as belonging to the range of whatever characters precede them. they're just matching the vibe of whoever came before, honestly relatable 😭

if the input begins with neutral characters, then they're treated as belonging to the range of whatever characters follow them. and if the whole file is nothing but neutral characters? then the range gets identified as Neutral followed by Basic Latin in square brackets. a whole file of pure neutral energy, we love to see it 🎀

---

now, the magic number tea ☕

a magic number identifying the Unicode encoding is not part of the Unicode standard, so pure Unicode files do not contain a magic number. however, informal conventions have arisen for this purpose (the girls made up their own rules, as usual).

if you give the command line flag -m, unidesc will attempt to identify the Unicode subtype by examining the first few bytes of the input. if the input is identified as one of the two acceptable types — UTF-8 or native order UTF-32 — she'll then proceed to describe the contents of the input. otherwise, she'll report what she has learned and exit. she's honest like that 💯

⚠️ okay listen: if the file does contain a magic number, you must use the -m flag. without this flag, unidesc assumes the input is pure Unicode with the character data beginning immediately — which means she will be thrown off by the magic number. do not do this to her. do not do this to yourself. 🙈

by default, input is expected to be UTF-8. native order UTF-32 is also acceptable. UTF-32 can be specified via the command line flag -u, or — if the command line flag -m is given — via the magic number. slay, options ✨

---

💅 COMMAND LINE FLAGS 💅

• -b — Give file offsets in bytes rather than characters.
• -d — Treat the ASCII digits as belonging exclusively to the Basic Latin range.
• -h — Print usage information.
• -L — List the Unicode ranges alphabetically.
• -l — List the Unicode ranges by codepoint.
• -m — Check the file's magic number to determine the Unicode subtype.
• -p — Treat ASCII punctuation as belonging exclusively to the Basic Latin range.
• -r — Instead of listing ranges as they are encountered, just list the ranges detected after all input has been read.
• -u — Input is native order UTF-32.
• -v — Print version information.
• -w — Treat ASCII whitespace as belonging exclusively to the Basic Latin range.

---

🦋 SEE ALSO 🦋

uniname

---

📚 REFERENCES 📚

Unicode Standard, version 5.0

---

🌈 AUTHOR 🌈

Bill Poser billposer@alum.mit.edu

---

💖 LICENSE 💖

GNU General Public License

---

and that's the whole thing bestie!! this page is from June, 2007, lives in section 1 of the manual as unidesc(1), and honestly? iconic behavior. go describe some Unicode. 🫶⭐