std::codecv...ecvt::do_in(3)

okay so bestie, this is page std::codecv...ecvt::do_in(3) and we're diving into std::codecvt::in and std::codecvt::do_in from the C++ Standard Library — it's giving character conversion realness and honestly I'm obsessed 💖✨

💖 NAME 💖

std::codecvt::in, std::codecvt::do_in

✨ Synopsis ✨

okay so first things first, this is defined in the header <locale> and here's the whole vibe:

public:

result in( StateT& state,
           const ExternT* from,
           const ExternT* from_end,              (1)
           const ExternT*& from_next,
           InternT* to,
           InternT* to_end,

           InternT*& to_next ) const;
protected:

virtual result do_in( StateT& state,
                    const ExternT* from,
                    const ExternT* from_end,   (2)
                    const ExternT*& from_next,
                    InternT* to,
                    InternT* to_end,

                    InternT*& to_next ) const;

1. So this public member function is basically the main character — it calls the do_in member function of the most derived class. slay.
2. And this one, if the codecvt facet defines a conversion, translates the external characters from the source range [from, from_end) to internal characters, placing the results in the subsequent locations starting at to. It converts no more than from_end - from external characters and writes no more than to_end - to internal characters. Leaves from_next and to_next pointing one beyond the last element successfully converted. periodt.

but wait — if this codecvt facet does not define a conversion, no characters are converted. to_next is set to be equal to to, state is unchanged, and std::codecvt_base::noconv is returned. it's giving "I'm not even gonna try" energy 😭

also do_in(state, from, from_end, from_next, to, to + 1, to_next) must return ok if:

• this codecvt facet is used by basic_filebuf, and
• do_in(state, from, from_end, from_next, to, to_end, to_next) would return ok where to != to_end.

🎀 Return value 🎀

a value of type std::codecvt_base::result, indicating the success status as follows:

• ok — conversion completed, we love to see it 💅
• partial — not enough space in the output buffer or unexpected end of source buffer, tragic
• error — encountered a character that could not be converted, big yikes
• noconv — this facet is non-converting, no output written, it's giving lazy but valid

also the non-converting specialization std::codecvt<char, char, std::mbstate_t> always returns std::codecvt_base::noconv. not me calling it out but okay.

🌸 Notes 🌸

requires that from <= from_end && to <= to_end and that state either representing the initial shift state or obtained by converting the preceding characters in the sequence. like, respect the boundaries bestie.

the effect on state is deliberately unspecified. in standard facets, it's used to maintain shift state like when calling std::mbsrtowcs, and is therefore updated to reflect the conversion state after the last processed external character, but a user-defined facet is free to use it to maintain any other state, e.g. count the number of special characters encountered. so it's giving "choose your own adventure" energy 🦋

🦋 Example 🦋

okay so here's the whole thing in action:

// Run this code

#include <iostream>
#include <locale>
#include <string>

int main()
{
    std::locale::global(std::locale("en_US.utf8"));
    auto const& f = std::use_facet<std::codecvt<wchar_t, char, std::mbstate_t>>
        (std::locale());
    std::string external = "z\u00df\u6c34\U0001d10b"; // or u8"zß水𝄋"
                     // or "\x7a\xc3\x9f\xe6\xb0\xb4\xf0\x9d\x84\x8b"

    // note that the following can be done with wstring_convert
    std::mbstate_t mb = std::mbstate_t(); // initial shift state
    std::wstring internal(external.size(), '\0');
    const char* from_next;
    wchar_t* to_next;
    f.in(mb, &external[0], &external[external.size()], from_next,
             &internal[0], &internal[internal.size()], to_next);
    // error checking skipped for brevity
    internal.resize(to_next - &internal[0]);

    std::wcout << L"The string in wide encoding: " << internal << '\n';
}

output:

The string in wide encoding: zß水𝄋

ate that conversion 💅

🔥 Defect reports 🔥

the following behavior-changing defect reports were applied retroactively to previously published C++ standards.

• DR LWG 76, applied to C++98 — the behavior as published was that it was unclear whether the conversion is required to support producing one internal character at a time. the correct behavior is that it's only required if used by basic_filebuf. it's giving "clarified and corrected" realness.

🌈 See also 🌈

• underflow — reads from the associated file [virtual] (virtual protected member function of std::basic_filebuf<CharT,Traits>)
• from_bytes — converts a byte string into a wide string (public member function of std::wstring_convert<Codecvt,Elem,Wide_alloc,Byte_alloc>)
• mbsrtowcs — converts a narrow multibyte character string to wide string, given state (function)
• do_out — converts a string from InternT to ExternT, such as when writing to file [virtual] (virtual protected member function)

---

and that's the whole vibe bestie 💖 this page is from cppreference.com and the version date is 2024.06.10 — iconic as always ✨