std::codecvt_utf8(3)

✨ NAME ✨

std::codecvt_utf8 — literally std::codecvt_utf8.

Okay bestie, welcome to std::codecvt_utf8(3) from the C++ Standard Libary (yes, that's the header's spelling, no cap 👀), and this page is all about the std::codecvt_utf8 facet. Let's get into it. 💅

🦋 Synopsis 🦋

Defined in header <codecvt>.

template<
    class Elem,                                         (since C++11)
    unsigned long Maxcode = 0x10ffff,                   (deprecated in C++17)
    std::codecvt_mode Mode = (std::codecvt_mode)0 >     (removed in C++26)
class codecvt_utf8
    : public std::codecvt<Elem, char, std::mbstate_t>;

std::codecvt_utf8 is a std::codecvt facet which encapsulates conversion between a UTF-8 encoded byte string and UCS-2 or UTF-32 character string (depending on the type of Elem). This std::codecvt facet can be used to read and write UTF-8 files, both text and binary. 💖

UCS-2 is the same encoding as UTF-16, except that it encodes scalar (since C++23) values in the range U+0000-U+FFFF (Basic Multilingual Plane) only. Yes, that (since C++23) is wedged right in there, and we're keeping it. 🫶

🌈 Template Parameters 🌈

• Elem — either char16_t, char32_t, or wchar_t.
• Maxcode — the largest value of Elem that this facet will read or write without error.
• Mode — a constant of type std::codecvt_mode.

🎀 Member functions 🎀

• constructor — constructs a new codecvt_utf8 facet (public member function).
• destructor — destroys a codecvt_utf8 facet (public member function).

💖 std::codecvt_utf8::codecvt_utf8 💖

explicit codecvt_utf8( std::size_t refs = 0 );

Constructs a new std::codecvt_utf8 facet, passes the initial reference counter refs to the base class.

Parameters

• refs — the number of references that link to the facet.

💀 std::codecvt_utf8::~codecvt_utf8 💀

~codecvt_utf8();

Destroys the facet. Unlike the locale-managed facets, this facet's destructor is public.

🧚 Inherited from std::codecvt 🧚

Member types

• intern_type → internT
• extern_type → externT
• state_type → stateT

Member objects

• id (static) — std::locale::id

Member functions

• out — invokes do_out (public member function of std::codecvt<InternT,ExternT,StateT>).
• in — invokes do_in (public member function of std::codecvt<InternT,ExternT,StateT>).
• unshift — invokes do_unshift (public member function of std::codecvt<InternT,ExternT,StateT>).
• encoding — invokes do_encoding (public member function of std::codecvt<InternT,ExternT,StateT>).
• always_noconv — invokes do_always_noconv (public member function of std::codecvt<InternT,ExternT,StateT>).
• length — invokes do_length (public member function of std::codecvt<InternT,ExternT,StateT>).
• max_length — invokes do_max_length (public member function of std::codecvt<InternT,ExternT,StateT>).

Protected member functions

• do_out [virtual] — converts a string from InternT to ExternT, such as when writing to file (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).
• do_in [virtual] — converts a string from ExternT to InternT, such as when reading from file (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).
• do_unshift [virtual] — generates the termination character sequence of ExternT characters for incomplete conversion (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).
• do_encoding [virtual] — returns the number of ExternT characters necessary to produce one InternT character, if constant (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).
• do_always_noconv [virtual] — tests if the facet encodes an identity conversion for all valid argument values (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).
• do_length [virtual] — calculates the length of the ExternT string that would be consumed by conversion into given InternT buffer (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).
• do_max_length [virtual] — returns the maximum number of ExternT characters that could be converted into a single InternT character (virtual protected member function of std::codecvt<InternT,ExternT,StateT>).

🫶 Inherited from std::codecvt_base 🫶

• Member type: enum result { ok, partial, error, noconv }; — unscoped enumeration type.

Enumeration constants:

• ok — conversion was completed with no error.
• partial — not all source characters were converted.
• error — encountered an invalid character.
• noconv — no conversion required, input and output types are the same.

👑 Notes 👑

• Although the standard requires that this facet works with UCS-2 when the size of Elem is 16 bits, some implementations use UTF-16 instead. The term "UCS-2" was deprecated and removed from ISO 10646.
• Since C++23, UCS-2 does not refer to its original definition anymore. It becomes a synonym of UTF-16, but with a smaller encoding range.

🌸 Example 🌸

The following example demonstrates the difference between UCS-2/UTF-8 and UTF-16/UTF-8 conversions: the third character in the string is not a valid UCS-2 character.

// Run this code

#include <codecvt>
#include <cstdint>
#include <iostream>
#include <locale>
#include <string>

int main()
{
    // UTF-8 data. The character U+1d10b, musical sign segno, does not fit in UCS-2
    std::string utf8 = "z\u6c34\U0001d10b";

    // the UTF-8 / UTF-16 standard conversion facet
    std::wstring_convert<std::codecvt_utf8_utf16<char16_t>, char16_t> utf16conv;
    std::u16string utf16 = utf16conv.from_bytes(utf8);
    std::cout << "UTF-16 conversion produced " << utf16.size() << " code units:\n"
              << std::showbase << std::hex;
    for (char16_t c : utf16)
        std::cout << static_cast<std::uint16_t>(c) << ' ';

    // the UTF-8 / UCS-2 standard conversion facet
    std::wstring_convert<std::codecvt_utf8<char16_t>, char16_t> ucs2conv;
    try
    {
        std::u16string ucs2 = ucs2conv.from_bytes(utf8);
    }
    catch(const std::range_error& e)
    {
        std::u16string ucs2 = ucs2conv.from_bytes(utf8.substr(0, ucs2conv.converted()));
        std::cout << "\nUCS-2 failed after producing " << std::dec << ucs2.size()
                  << " characters:\n" << std::showbase << std::hex;
        for (char16_t c : ucs2)
            std::cout << static_cast<std::uint16_t>(c) << ' ';
        std::cout << '\n';
    }
}

🍓 Output 🍓

UTF-16 conversion produced 4 code units:
0x7a 0x6c34 0xd834 0xdd0b
UCS-2 failed after producing 2 characters:
0x7a 0x6c34

⭐ See also ⭐

So the original page has a big character-conversions table. Its column labels are Character conversions, locale-defined multibyte (UTF-8, GB18030), UTF-8, and UTF-16; its row labels are UTF-16, UCS-2, UTF-32, and system wchar_t: UTF-32 (non-Windows) UCS-2 (Windows). Every reference from it, no cap:

• mbrtoc16 / c16rtomb (with C11's DR488)
• codecvt_utf8_utf16<char16_t>
• codecvt_utf8_utf16<char32_t>
• codecvt_utf8_utf16<wchar_t>
• codecvt<char16_t,char,mbstate_t>
• N/A
• c16rtomb (without C11's DR488)
• codecvt_utf8<char16_t>
• codecvt_utf16<char16_t>
• mbrtoc32 / c32rtomb
• codecvt<char32_t,char,mbstate_t>
• codecvt_utf16<char32_t>
• codecvt_utf8<char32_t>
• mbsrtowcs / wcsrtombs
• use_facet<codecvt<wchar_t,char,mbstate_t>>(locale)
• codecvt_utf8<wchar_t>
• codecvt_utf16<wchar_t>

And the related class/enum references:

• codecvt — converts between character encodings, including UTF-8, UTF-16, UTF-32 (class template).
• codecvt_mode (C++11) (deprecated in C++17) (removed in C++26) — tags to alter behavior of the standard codecvt facets (enum).
• codecvt_utf16 (C++11) (deprecated in C++17) (removed in C++26) — converts between UTF-16 and UCS-2/UCS-4 (class template).
• codecvt_utf8_utf16 (C++11) (deprecated in C++17) (removed in C++26) — converts between UTF-8 and UTF-16 (class template).

---

Made with 💖 from http://cppreference.com, version/date 2024.06.10 — that's the footer, bestie, and yes, std::codecvt_utf8(3) is iconic even when it's deprecated. 🎀✨