ICONV(3P)

💖 ICONV(3P) — POSIX Programmer's Manual 💖

ok so bestie, before we even get into it — this one is from the POSIX Programmer's Manual. Quick lil heads up:

🌸 PROLOG 🌸

This manual page is part of the POSIX Programmer's Manual. The Linux implementation of this interface may differ (so like, go check the corresponding Linux manual page for the deets on Linux behavior), or honestly the interface may not be implemented on Linux at all. 🙈

---

✨ NAME ✨

iconv — codeset conversion function

the way this is literally the glow-up function for character encodings, iykyk 💅

🎀 SYNOPSIS 🎀

#include <iconv.h>

size_t iconv(iconv_t cd, char **restrict inbuf,
    size_t *restrict inbytesleft, char **restrict outbuf,
    size_t *restrict outbytesleft);

🦋 DESCRIPTION 🦋

Okay so here's the vibe: iconv() converts a sequence of characters from one codeset — the one sitting in the array pointed to by inbuf — into a sequence of corresponding characters in another codeset, in the array pointed to by outbuf. The codesets in question? Those are the ones you picked in the iconv_open() call that handed you back the conversion descriptor, cd. 👑

So inbuf points to a variable that points to the first character in the input buffer, and inbytesleft tells you how many bytes there are from there to the end of the buffer that still need converting. Meanwhile outbuf points to a variable that points to the first available byte in the output buffer, and outbytesleft is how many available bytes you've got from there to the end of that buffer.

For state-dependent encodings, the conversion descriptor cd gets placed into its initial shift state by a call where inbuf is a null pointer, or where inbuf points to a null pointer. When you call iconv() like that, and if outbuf is not a null pointer or a pointer to a null pointer, and outbytesleft points to a positive value, then iconv() shall place into the output buffer the byte sequence needed to change the output buffer to its initial shift state. But — and this is the important part — if the output buffer isn't big enough to hold that entire reset sequence, iconv() shall fail and set errno to [E2BIG]. 😭 Subsequent calls where inbuf is anything other than a null pointer or a pointer to a null pointer make the conversion happen from the current state of the conversion descriptor.

Now, the messy parts, and there are a few:

• If a sequence of input bytes doesn't form a valid character in the specified codeset, conversion shall stop after the previous successfully converted character.
• If the input buffer ends with an incomplete character or shift sequence, conversion shall stop after the previous successfully converted bytes.
• If the output buffer isn't large enough to hold the entire converted input, conversion shall stop just before the input bytes that would cause the output buffer to overflow.

And after all that, the bookkeeping goes like this:

• The variable pointed to by inbuf shall be updated to point to the byte following the last byte successfully used in the conversion.
• The value pointed to by inbytesleft shall be decremented to reflect the number of bytes still not converted in the input buffer.
• The variable pointed to by outbuf shall be updated to point to the byte following the last byte of converted output data.
• The value pointed to by outbytesleft shall be decremented to reflect the number of bytes still available in the output buffer.
• For state-dependent encodings, the conversion descriptor shall be updated to reflect the shift state in effect at the end of the last successfully converted byte sequence.

And one more thing: if iconv() encounters a character in the input buffer that is valid, but for which an identical character does not exist in the target codeset, iconv() shall perform an implementation-defined conversion on that character. (yes bestie, implementation-defined, meaning don't come crying to POSIX about it 🌈)

💜 RETURN VALUE 💜

iconv() shall update the variables pointed to by the arguments to reflect how far the conversion got, and it returns the number of non-identical conversions performed.

• If the entire string in the input buffer is converted, the value pointed to by inbytesleft shall be 0.
• If the input conversion is stopped for any of the conditions mentioned above, the value pointed to by inbytesleft shall be non-zero and errno shall be set to indicate the condition.
• If an error occurs, iconv() shall return (size_t)-1 and set errno to indicate the error.

🔥 ERRORS 🔥

The iconv() function shall fail if:

• EILSEQ — Input conversion stopped due to an input byte that does not belong to the input codeset.
• E2BIG — Input conversion stopped due to lack of space in the output buffer.
• EINVAL — Input conversion stopped due to an incomplete character or shift sequence at the end of the input buffer.

The iconv() function may fail if:

• EBADF — The cd argument is not a valid open conversion descriptor.

The following sections are informative. (so like, chill, these are bonus content, not the rules-rules 📝)

🌈 EXAMPLES 🌈

None. not a single one. iconic behavior honestly 💀

🫶 APPLICATION USAGE 🫶

ok so this part is lowkey the most important vibe check of the whole page:

The inbuf argument indirectly points to the memory area containing the conversion input data. The outbuf argument indirectly points to the memory area that's gonna hold the result of the conversion. And here's the thing — the objects indirectly pointed to by inbuf and outbuf are not restricted to containing data that's directly representable in the ISO C standard language char data type. The type of inbuf and outbuf, char **, does not imply that the objects pointed to are interpreted as null-terminated C strings or arrays of characters. 🚨

Any interpretation of a byte sequence that represents a character in a given character set encoding scheme happens internally within the codeset converters. For example, the area pointed to indirectly by inbuf and/or outbuf can contain all zero octets that are not interpreted as string terminators but as coded character data according to the respective codeset encoding scheme. The type of the data (char, short, long, and so on) read or stored in the objects is not specified, but it may be inferred for both the input and output data by the converters determined by the fromcode and tocode arguments of iconv_open().

And regardless of whatever data type the converter inferred, the size of the remaining space in both the input and output objects — the intbytesleft and outbytesleft arguments — is always measured in bytes. always. no exceptions. 📏

For implementations that support converting state-dependent encodings, the conversion descriptor must be able to accurately reflect the shift-state in effect at the end of the last successful conversion. It is not required that the conversion descriptor itself be updated — because that would require it to be a pointer type. So implementations are free to implement the descriptor as a handle (something other than a pointer type) through which the conversion information can be accessed and updated. cute and flexible of them ✨

💅 RATIONALE 💅

None. (the reasoning is simply that it slays, periodt)

⭐ FUTURE DIRECTIONS ⭐

None.

🌸 SEE ALSO 🌸

• iconv_open()
• iconv_close()
• mbsrtowcs()
• The Base Definitions volume of POSIX.1‐2017, <iconv.h>

👑 COPYRIGHT 👑

Portions of this text are reprinted and reproduced in electronic form from IEEE Std 1003.1-2017, Standard for Information Technology -- Portable Operating System Interface (POSIX), The Open Group Base Specifications Issue 7, 2018 Edition, Copyright (C) 2018 by the Institute of Electrical and Electronics Engineers, Inc and The Open Group. In the event of any discrepancy between this version and the original IEEE and The Open Group Standard, the original IEEE and The Open Group Standard is the referee document. The original Standard can be obtained online at http://www.opengroup.org/unix/online.html .

Any typographical or formatting errors that appear in this page are most likely to have been introduced during the conversion of the source files to man page format. To report such errors, see https://www.kernel.org/doc/man-pages/reporting_bugs.html .

---

and that's the whole thing bestie 🫶 this page came to you from IEEE/The Open Group, 2017, and the original page header was ICONV(3P) — go forth and convert some codesets, you've got this 💖✨🎀