UNICODE_BIDI(3)

📦

version 2.6.0 · 8 versions

💖 UNICODE_BIDI(3) 💖

okay bestie, sit DOWN, because this is the Courier Unicode Library's whole bi-directional algorithm situation and it is a lot in the best way 💅 this is unicode_bidi(3) and it's giving main character energy for real.

✨ NAME ✨

Here's the full squad, no one gets left out 🫶

unicode_bidi
unicode_bidi_calc_levels
unicode_bidi_calc_types
unicode_bidi_calc
unicode_bidi_reorder
unicode_bidi_cleanup
unicode_bidi_cleaned_size
unicode_bidi_logical_order
unicode_bidi_combinings
unicode_bidi_needs_embed
unicode_bidi_embed
unicode_bidi_embed_paragraph_level
unicode_bidi_direction
unicode_bidi_type
unicode_bidi_setbnl
unicode_bidi_mirror
unicode_bidi_bracket_type

…and what are they all about? unicode bi-directional algorithm ✨ periodt.

🌸 SYNOPSIS 🌸

#include <courier-unicode.h>

unicode_bidi_level_t lr=UNICODE_BIDI_LR;

void unicode_bidi_calc_types(const char32_t *p, size_t n,
                             unicode_bidi_type_t *types);

struct unicode_bidi_direction
                                                       unicode_bidi_calc_levels(const char32_t *p,
                                                       const unicode_bidi_type_t *types,
                                                       size_t n,
                                                       unicode_bidi_level_t *levels,
                                                       const unicode_bidi_level_t *initial_embedding_level);

struct unicode_bidi_direction unicode_bidi_calc(const char32_t *p,
                                                size_t n,
                                                unicode_bidi_level_t *levels,
                                                const unicode_bidi_level_t *initial_embedding_level);

void unicode_bidi_reorder(char32_t *string,
                          unicode_bidi_level_t *levels, size_t n,
                          void (*reorder_callback)(size_t, size_t, void *),
                          void *arg);

size_t unicode_bidi_cleanup(char32_t *string,
                            unicode_bidi_level_t *levels, size_t n,
                            int options,
                            void (*removed_callback)(size_t, size_t, void *),
                            void *arg);

size_t unicode_bidi_cleaned_size(const char32_t *string, size_t n,
                                 int options);

void unicode_bidi_logical_order(char32_t *string,
                                unicode_bidi_level_t *levels, size_t n,
                                unicode_bidi_level_t paragraph_embedding,
                                void (*reorder_callback)(size_t index, size_t n, void *arg),
                                void *arg);

void unicode_bidi_combinings(const char32_t *string,
                             const unicode_bidi_level_t *levels,
                             size_t n,
                             void (*combinings)(unicode_bidi_level_t
                             level, size_t level_start, size_t n_chars,
                             size_t comb_start, size_t n_comb_chars, void *arg),
                             void *arg);

int unicode_bidi_needs_embed(const char32_t *string, size_t n,
                             const unicode_bidi_level_t *paragraph_embedding);

size_t unicode_bidi_embed(const char32_t *string,
                          const unicode_bidi_level_t *levels, size_t n,
                          unicode_bidi_level_t paragraph_embedding, void
                          (*emit)(const char32_t *string, size_t n, int is_part_of_string, void *arg),
                          void *arg);

char32_t unicode_bidi_embed_paragraph_level(const char32_t *string,
                                            size_t n,
                                            unicode_bidi_level_t paragraph_embedding);

char32_t bidi_mirror(char32_t c);

char32_t bidi_bracket_type(char32_t c, unicode_bracket_type_t *ret);

struct unicode_bidi_direction unicode_bidi_get_direction(char32_t *c,
                                                         size_t n);

enum_bidi_type_t unicode_bidi_type(char32_t c);

void unicode_bidi_setbnl(char32_t *p, const unicode_bidi_type_t *types,
                         size_t n);

🦋 DESCRIPTION 🦋

So these functions are all about the Unicode Bi-Directional algorithm[1] 💜 They implement that algorithm up to and including step L2, and on top of that they hand you a bunch of miscellaneous bi-directional-related metadata about Unicode characters. And there's also a basic algorithm that basically "reverses" the bi-directional algorithm — it produces a Unicode string with bi-directional markers that, after you reapply the algorithm, gives you back the same bi-directional string. iconic, honestly.

🧮 Calculating bi-directional rendering order

Here's the whole ritual for computing the rendering order of characters per the Unicode Bi-Directional algorithm, step by step, no skipping 🫶

Allocate an array of unicode_bidi_type_t that's the same size as the Unicode string.
Allocate an array of unicode_bidi_level_t that's the same size as the Unicode string.
Use unicode_bidi_calc_types() to compute the Unicode string's characters' bi-directional types, and populate the unicode_bidi_type_t buffer.
Use unicode_bidi_calc_levels() to compute the Unicode string's characters' bi-directional embedding level (that's the Bi-Directional algorithm running up to and including step L1). This populates the unicode_bidi_level_t buffer.
Alternatively: allocate only the unicode_bidi_level_t array and use unicode_bidi_calc(), which malloc()s the unicode_bidi_type_t buffer, calls unicode_bidi_calc_levels(), and then free()s the buffer.
Use unicode_bidi_reorder() to reverse any characters in the string, according to the algorithm (step L2), with an optional callback that reports which ranges of characters get reversed.
Use unicode_bidi_cleanup() to remove the characters from the string which are used by the bi-directional algorithm, and are not needed for rendering the text. unicode_bidi_cleaned_size() is available to determine, in advance, how many characters will remain.

The parameters to unicode_bidi_calc_types() are:

A pointer to the Unicode string.
Number of characters in the Unicode string.
A pointer to an array of unicode_bidi_type_t values. The caller is responsible for allocating and deallocating this array, which has the same size as the Unicode string.

The parameters to unicode_bidi_calc_levels() are:

A pointer to the Unicode string.
A pointer to the buffer that was passed to unicode_bidi_calc_types().
Number of characters in the Unicode string and the unicode_bidi_type_t buffer.
A pointer to an array of unicode_bidi_level_t values. The caller is responsible for allocating and deallocating this array, which has the same size as the Unicode string.
An optional pointer to a UNICODE_BIDI_LR or UNICODE_BIDI_RL value. This sets the default paragraph direction level. A null pointer computes the default paragraph direction level based on the string, as specified by the "P" rules of the bi-directional algorithm.

The parameters to unicode_bidi_calc() are the same except for the unicode_bidi_type_t pointer. unicode_bidi_calc() allocates this buffer by itself and calls unicode_bidi_calc_types, and destroys the buffer before returning.

unicode_bidi_calc() and unicode_bidi_calc_levels() fill in the unicode_bidi_level_t array with the values corresponding to the embedding level of the corresponding character, according the Unicode Bidirection Algorithm (even values for left-to-right ordering, and odd values for right-to-left ordering). A value of UNICODE_BIDI_SKIP designates directional markers (from step X9).

unicode_bidi_calc() and unicode_bidi_calc_levels() return the resolved paragraph direction level, which always matches the passed in level, if specified, else it reports the derived one. These functions return a unicode_bidi_direction structure:

struct unicode_bidi_direction {
    unicode_bidi_level_t   direction;
    int                    is_explicit;
};

direction gives the paragraph embedding level, UNICODE_BIDI_LR or UNICODE_BIDI_RL. is_explicit indicates whether: the optional pointer to a UNICODE_BIDI_LR or UNICODE_BIDI_RL value was specified (and returned in direction), or whether the direction comes from an character with an explicit direction indication.

unicode_bidi_reorder() takes the actual unicode string together with the embedding values from unicode_bidi_calc or unicode_bidi_calc_levels(), then reverses the bi-directional string, as specified by step L2 of the bi-directional algorithm. The parameters to unicode_bidi_reorder() are:

A pointer to the Unicode string.
A pointer to an array of unicode_bidi_level_t values.
Number of characters in the Unicode string and the unicode_bidi_level_t array.
An optional reorder_callback function pointer.

A non-NULL reorder_callback gets invoked to report each reversed character range. The callback's first parameter is the index of the first reversed character, the second parameter is the number of reversed characters, starting at the given index of the Unicode string. The third parameter is the arg passthrough parameter.

unicode_bidi_reorder modifies its string and levels. reorder_callback gets invoked after reversing each consecutive range of values in the string and levels buffers. For example: “reorder_callback(5, 7, arg)” reports that character indexes #5 through #11 got reversed. 💫

A NULL string pointer leaves the levels buffer unchanged, but still invokes the reorder_callback as if the character string, and their embedding values, were reversed. no cap.

The resulting string and embedding levels are in “rendering order”, but still contain bi-directional embedding, override, boundary-neutral, isolate, and marker characters. unicode_bidi_cleanup removes these characters and directional markers.

The parameters to unicode_bidi_cleanup() are:

The pointer to the unicode string.
A non-null pointer to the directional embedding level buffer, of the same size as the string, also removes the corresponding values from the buffer, and the remaining values in the embedding level buffer get reset to levels UNICODE_BIDI_LR and UNICODE_BIDI_RL, only.
The size of the unicode string and the directional embedding buffer (if not NULL).
A a bitmask that selects the following options (or 0 if no options):
- UNICODE_BIDI_CLEANUP_EXTRA — In addition to removing all embedding, override, and boundry-neutral characters as specified by step X9 of the bi-directional algorithm (that's the default behavior without this flag), this also removes all isolation markers and implicit markers.
- UNICODE_BIDI_CLEANUP_BNL — Replace all characters classified as paragraph separators with a newline character.

<div align="center">💖 ✨ 🎀 more coming, don't go anywhere bestie 🎀 ✨ 💖</div>

UNICODE_BIDI_CLEANUP_CANONICAL — bestie, this is literally just UNICODE_BIDI_CLEANUP_EXTRA and UNICODE_BIDI_CLEANUP_BNL holding hands. A combined set, dynamic duo, no cap. 👑
A pointer to a function that gets invoked again and again, each time with the index of the character that gets yeeted right out of the Unicode string. 🔥
An opaque pointer that just gets forwarded straight to the callback. Not our business what's in there. 🙈

That function pointer (NULL is fine, it'll just sit there quietly) gets invoked to report the index of every removed character. The index it reports is the index from the original string, and the callbacks come in strict order — from the first removed character to the last one (if there even are any). Periodt. 💅

The character string and the embedding level values you get back from unicode_bidi_cleanup() when you used the UNICODE_BIDI_CLEANUP_CANONICAL option are in “canonical rendering order”. And heads up: unicode_bidi_logical_order(), unicode_bidi_needs_embed() and unicode_bidi_embed() all require the canonical rendering order for their string and embedding level values. It's the vibe check they won't pass without. ✨

The parameters to unicode_bidi_cleaned_size() are a pointer to the unicode string, its size, and the bitmask option to unicode_bidi_cleanup(). That's the whole squad. 🫶

✨ Embedding bi-directional markers in Unicode text strings ✨

Okay so unicode_bidi_logical_order() takes the string and rearranges it from rendering order back to its logical order. Meanwhile unicode_bidi_embed() sprinkles all kinds of bi-directional markers onto a Unicode string that's sitting in canonical rendering order. 🦋

And ngl, the resulting string is not guaranteed to be identical to the original Unicode bi-directional string. It's not a copy situation. But — and this is the slay part — the resulting bi-directional string still produces the same canonical rendering order after you apply unicode_bidi_calc() or unicode_bidi_calc_levels(), then unicode_reorder(), then unicode_bidi_cleanup() (with the canonical option), all with the same paragraph_embedding level. Same energy, different spelling. 💖

Also, unicode_bidi_needs_embed() attempts to heuristically determine whether unicode_bidi_embed() is even required. It's guessing, but like, an educated guess. 🤔

Here's the order of operations, bestie:

unicode_bidi_logical_order() gets called first.
Then unicode_bidi_embed() — or unicode_bidi_needs_embed() if you just wanna know whether bi-directional markers are required at all.
Finally, unicode_bidi_embed_paragraph_level() optionally determines whether the resulting string's default paragraph embedding level matches the one used for the actual embedding direction. If it doesn't match, it returns a directional marker that you prepend to the Unicode character string, as a hint. 🎀

unicode_bidi_logical_order() factors in the characters' embedding values and the provided paragraph embedding value (UNICODE_BIDI_LR or UNICODE_BIDI_RL), and rearranges the characters and the embedding levels into left-to-right order — while simultaneously invoking the supplied reorder_callback to flag every range of characters whose relative order gets reversed. 🔄

The reorder_callback() receives, as parameters:

The starting index of the first reversed character, in the string.
Number of reversed characters.
Forwarded arg pointer value.

This specifies a consecutive range of characters (and directional embedding values) that get reversed — first character in the range becomes the last character, and the last character becomes the first character. Full flip. 🙃

After unicode_bidi_logical_order(), unicode_bidi_embed() progressively invokes the passed-in callback with the contents of a bi-directional unicode string. The parameters to unicode_bidi_embed() are:

The Unicode string.
The directional embedding buffer, in canonical rendering order.
The size of the string and the embedding level buffer.
The paragraph embedding level, either UNICODE_BIDI_LR or UNICODE_BIDI_RL.
The pointer to the callback function.
An opaque pointer argument that gets forwarded to the callback function.

The callback receives pointers to various parts of the original string that gets passed to unicode_bidi_embed(), intermixed with bi-directional markers, overrides, and isolates. Its parameters are:

The pointer to a Unicode string.

Note — it is not a given that the callback receives pointers that progressively increase through the original string passed to unicode_bidi_embed(). Some calls will be for individual bi-directional markers, and unicode_bidi_embed() also performs some additional internal reordering, on the fly, after unicode_bidi_logical_order()'s big hammer. So don't go assuming the addresses line up neatly. 😭

Number of characters in the Unicode string.
An indication whether the Unicode string pointer is pointing to a part of the original Unicode string that's getting embedded. Otherwise this must be some marker character that's not present in the original Unicode string.
Forwarded arg pointer value.

The assembled unicode string should produce the same canonical rendering order, for the same paragraph embedding level. That's the goal, that's the dream. 🌈

unicode_bidi_embed_paragraph_level() checks if the specified Unicode string computes the given default paragraph embedding level and returns 0 if it matches. Otherwise it returns a directional marker that should be prepended to the Unicode string — so that unicode_bidi_calc's (or unicode_bidi_calc_levels()) optional paragraph embedding level pointer's value can be NULL, while still deriving the same default embedding level. Clever, honestly. 👑

The parameters to unicode_bidi_embed_paragraph_level() are:

The Unicode string.
The size of the string.
The paragraph embedding level, either UNICODE_BIDI_LR or UNICODE_BIDI_RL.

unicode_bidi_needs_embed() attempts to heuristically determine whether the Unicode string, in logical order, requires bi-directional markers. The parameters to unicode_bidi_embed_paragraph_level() are:

(yes bestie, the original says unicode_bidi_embed_paragraph_level() right here — copying it exactly, this is what the manpage says, I'm not gonna editorialize 😭)

The Unicode string.
The directional embedding buffer, in logical order.
The size of the string and the embedding level buffer.
A pointer to an explicit paragraph embedding level, either UNICODE_BIDI_LR or UNICODE_BIDI_RL; or a NULL pointer (see unicode_bidi_calc_types()'s explanation for this parameter).

unicode_bidi_needs_embed() returns 0 if the Unicode string does not need explicit directional markers, or 1 if it does. How does it know? It uses unicode_bidi_calc(), unicode_bidi_reorder(), unicode_bidi_logical_order and then checks if the end result is different from what was passed in. Detective work. 🔍

✨ Combining character ranges ✨

unicode_bidi_combinings() reports consecutive sequences of one or more combining marks in bidirectional text (which can be either in rendering or logical order) that have the same embedding level. It takes the following parameters:

The Unicode string.
The directional embedding buffer, in logical or rendering order. A NULL value for this pointer is equivalent to a directional embedding buffer with a level of 0 for every character in the Unicode string.
Number of characters in the Unicode string.
The pointer to the callback function.
An opaque pointer argument that gets forwarded to the callback function.

The callback function gets invoked for every consecutive sequence of one or more characters that have a canonical combining class other than 0, and with the same embedding level. The parameters to the callback function are:

The embedding level of the combining characters.
The starting index of a consecutive sequence of all characters with the same embedding level.
The number of characters with the same embedding level.
The starting index of a consecutive sequence of all characters with the same embedding level and a canonical combining class other than 0. This will always be equal to or greater than the value of the second parameter.
The number of consecutive characters with the same embedding level and a canonical combining class other than 0. The last character included in this sequence will always be less than or equal to the last character in the sequence defined by the second and the third parameters.
The opaque pointer argument that was passed to unicode_bidi_combinings.

A consecutive sequence of Unicode characters with non-0 combining classes but different embedding levels gets reported individually, for each consecutive sequence with the same embedding level. So they don't get lumped together, nope. 🫣

This function helps with reordering the combining characters in right-to-left-rendered text. Here's the tea: right-to-left text reversed by unicode_bidi_reorder() ends up with combining characters preceding their starter character. They get reversed no differently than any other character. The same thing also occurs after unicode_bidi_logical_order() reverses everything back. So use unicode_bidi_combinings to identify consecutive sequences of combining characters followed by their original starter. 🎀

The callback may reorder the characters identified by its third and the fourth parameters in the manner described below. unicode_bidi_reorder's parameter is pointers to a constant Unicode string — but it can still modify the string (via an out-of-band mutable pointer) subject to the following conditions:

The characters identified by the third and the fourth parameter may be modified.
If the last character in this sequence is not the last character included in the range specified by the first and the second character, then one more character after the last character may also be modified.
- This is, presumably, the original starter that preceded the combining characters before the entire sequence was reversed. 🧐

Here's an example of a callback that reverses combining characters and their immediately-following starter character:

void reorder_right_to_left_combining(unicode_bidi_level_t level,
                                     size_t level_start,
                                     size_t n_chars,
                                     size_t comb_start,
                                     size_t n_comb_chars,
                                     void *arg)
{
    /* Let's say that this is the Unicode string */
    char32_t *buf=(char32_t *)arg;

    if ((level & 1) == 0)
        return; /* Left-to-right text not reversed */

    char32_t *b=buf+comb_start;
    char32_t *e=b+n_comb_chars;

    /*
    ** Include the starter characters in the reversed range.
    ** The semantics of the combining characters with different
    ** embedding levels -- so they get reported here separately -- is
    ** not specified. This will reverse just the combining marks, and
    ** they're on their own.
    */

    if (comb_start + n_comb_chars < level_start + n_chars)
        ++e;

    while (b < e)
    {
        char32_t t;

...and that's the whole swap-situation, bestie, the closing braces just vibing there. 🎀

🌈 Miscellaneous utility functions 🌈

💖 `unicode_bidi_get_direction` 💖

So it takes a pointer to a unicode string, the number of characters in that unicode string, and it figures out the default paragraph level for you. Slay.

It hands back a struct, and the struct's got these fields, no cap:

direction — this value is either UNICODE_BIDI_LR or UNICODE_BIDI_RL (that's left to right or right to left, iykyk).
is_explicit — this value is a flag. A non-0 value means the embedding level got derived from an explicit character type (L, R or AL) that was sitting there in the string. A 0 value means it's just the default paragraph direction — no explicit character was found in the string at all. Periodt.

🦋 `unicode_bidi_type` 🦋

This one literally just looks up each character's bi-directional character type. That's the whole job. Main character behavior honestly. ✨

🌸 `unicode_bidi_setbnl` 🌸

Takes a pointer to a unicode string, a pointer to an array of enum_bidi_type_t values, and the number of characters in the string and the array. Then it goes through and replaces all paragraph separators in the unicode string with a newline character.

Fyi bestie, this is exactly the same thing the UNICODE_BIDI_CLEANUP_BNL option does over in unicode_bidi_cleanup. Same energy, different door. 🚪✨

🧚 `unicode_bidi_mirror` 🧚

Returns the glyph that's the mirror image of the parameter you passed in — like, an open parenthesis for a close parenthesis, and vice versa. And if there's no mirror image? It just gives you back the same value. That's the Bidi_Mirrored=Yes property, for the record. 🤝

👑 `unicode_bidi_bracket_type` 👑

Looks up each bracket character and returns its opposite, or the same value if the character isn't a bracket that has an opposing bracket character. This one's the Bidi_Paired_Bracket_type property, so don't forget it.

And okay, important detail: a non-NULL ret gets initialized to either UNICODE_BIDI_o, UNICODE_BIDI_c or UNICODE_BIDI_n. Don't sleep on that. 💅

💜 SEE ALSO 💜

🫶 AUTHOR 🫶

Sam Varshavchik

Author

⭐ NOTES ⭐

Unicode Bi-Directional algorithm https://www.unicode.org/reports/tr9/tr9-51.html

okay we made it to the end and I'm not crying you're crying 😭💖 this whole page is from the Courier Unicode Library, dated 04/12/2026, and yeah, it's UNICODE_BIDI(3). go forth and be bidi-correct, bestie, you absolutely ate this one 👑🌈🍓

📦 where this page comes from

ubuntu stonking: libcourier-unicode-dev 2.6.0-2ubuntu1 · GPL-3.0-only
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms