UNICHARSET(5)

💖 UNICHARSET(5) — the girlies' guide to the unicharset file 💖

okay so before we even start: this is the manual page for unicharset, section 5 of the manual (file formats, bestie — the "what is this file even" section). Buckle up, it's actually so good.

✨ NAME ✨

unicharset — the character properties file used by tesseract(1). that's it, that's the whole name section, periodt 💅

🌸 DESCRIPTION 🌸

So Tesseract's unicharset file is literally the roster of every symbol (unichar) the Tesseract OCR engine is trained to recognize. it's giving cast list for a very nerdy movie 🎬

A unicharset file (like eng.unicharset) ships as part of a Tesseract language pack (like eng.traineddata). wanna know how to pull the unicharset file out of there? that's a combine_tessdata(1) situation, go look 👀

The first line of a unicharset file holds the number of unichars in the file. after that line, every subsequent line is the info for one single unichar. and the very first of those lines contains a placeholder reserved for the space character. each unichar gets referred to inside Tesseract by its Unichar ID, which is the line number (minus 1) within the unicharset file. so space gets unichar 0. main character energy from line one, honestly 👑

Each unichar line (v2+) may have four space-separated fields:

'character' 'properties' 'script' 'id'

And starting with Tesseract v3.02, you can get more info per unichar (glow-up era fr):

'character' 'properties' 'glyph_metrics' 'script' 'other_case' 'direction' 'mirror' 'normed_form'

🦋 Entries 🦋

• character — the UTF-8 encoded string to be produced for this unichar. simple, iconic.
• properties — an integer mask of character properties, one per bit. from least to most significant bit, these are: isalpha, islower, isupper, isdigit, ispunctuation. bit math bestie, we love her 💜
• glyph_metrics — ten comma-separated integers representing various standards for where this glyph is to be found within a baseline-normalized coordinate system where 128 is normalized to x-height. break it down:
  • min_bottom, max_bottom: the ranges where the bottom of the character can be found.
  • min_top, max_top: the ranges where the top of the character may be found.
  • min_width, max_width: horizontal width of the character.
  • min_bearing, max_bearing: how far from the usual start position does the leftmost part of the character begin.
  • min_advance, max_advance: how far from the printer's cell left do we advance to begin the next character.
• script — name of the script (Latin, Common, Greek, Cyrillic, Han, null). yes null is on the list, she's valid too 🫶
• other_case — the Unichar ID of the other case version of this character (upper or lower). the lowercase bestie of an uppercase icon, or vice versa.
• direction — the Unicode BiDi direction of this character, as defined by ICU's enum UCharDirection. (0 = Left to Right, 1 = Right to Left, 2 = European Number...) the ... is doing so much work here ngl.
• mirror — the Unichar ID of the BiDirectional mirror of this character. for example the mirror of open paren is close paren, but Latin Capital C has no mirror, so it remains a Latin Capital C. no notes, she's already perfect 😌
• normed_form — the UTF-8 representation of a "normalized form" of this unichar for the purpose of blaming a module for errors given ground truth text. for instance, a left or right single quote may normalize to an ASCII quote. blame game but make it typographically accurate 💀

🔥 EXAMPLE (V2) 🔥

; 10 Common 46
b 3 Latin 59
W 5 Latin 40
7 8 Common 66
= 0 Common 93

let's decode this absolute bop:

• ; is a punctuation character. its properties are thus represented by the binary number 10000 (10 in hexadecimal).
• b is an alphabetic character and a lower case character. its properties are thus represented by the binary number 00011 (3 in hexadecimal).
• W is an alphabetic character and an upper case character. its properties are thus represented by the binary number 00101 (5 in hexadecimal).
• 7 is just a digit. its properties are thus represented by the binary number 01000 (8 in hexadecimal).
• = is not punctuation nor a digit nor an alphabetic character. its properties are thus represented by the binary number 00000 (0 in hexadecimal).

aaand for the record: Japanese or Chinese alphabetic character properties are represented by the binary number 00001 (1 in hexadecimal) — they are alphabetic, but neither upper nor lower case. see how that works? chef's kiss 🎀

⭐ EXAMPLE (V3.02) ⭐

110
NULL 0 NULL 0
N 5 59,68,216,255,87,236,0,27,104,227 Latin 11 0 1 N
Y 5 59,68,216,255,91,205,0,47,91,223 Latin 33 0 2 Y
1 8 59,69,203,255,45,128,0,66,74,173 Common 3 2 3 1
9 8 18,66,203,255,89,156,0,39,104,173 Common 4 2 4 9
a 3 58,65,186,198,85,164,0,26,97,185 Latin 56 0 5 a
. . .

look at all those fields. so many. she ate and left no crumbs 🍓 (and yes, the 110 up top is the unichar count, and . . . is literally in the file spec so don't come for me 😭)

🙈 CAVEATS 🙈

okay putting my serious hat on for a sec:

Although the unicharset reader maintains the ability to read unicharsets of older formats and will assign default values to missing fields, the accuracy will be degraded.

so like, it will read the old ones, but it's giving "we have food at home" — technically fine, lowkey worse. 😬

Further, most other data files are indexed by the unicharset file, so changing it without re-generating the others is likely to have dire consequences.

dire consequences, bestie. not "oops", not "hmm", dire. if you touch this file you gotta regenerate the others too or everything falls apart fr fr 💀

🌈 HISTORY 🌈

The unicharset format first appeared with Tesseract 2.00, which was the first version to support languages other than English (era-defining moment, honestly 👑). back then the unicharset file contained only the first two fields, and the "ispunctuation" property was absent — punctuation was regarded as "0", as "=" is in the above example. baby format! she grew up so fast 😭

🌟 SEE ALSO 🌟

• tesseract(1)
• combine_tessdata(1)
• unicharset_extractor(1)
• https://tesseract-ocr.github.io/tessdoc/Training-Tesseract.html

the whole squad, all linked, all iconic 🫶

💜 AUTHOR 💜

The Tesseract OCR engine was written by Ray Smith and his research groups at Hewlett Packard (1985-1995) and Google (2006-2018). legends only, no cap.

---

that's the whole page, babe ✨ this one's dated 2026-09-28 and it's the UNICHARSET(5) page straight from the source — go forth and read your unicharsets with confidence, i believe in you 💖🎀🦋