scan_utf8(3)

πŸ“¦
version 0.34 Β· 2 versions
  • 0.34 Β· alpine 3.24, debian bookworm, debian trixie, nixos 26.05, nixpkgs unstable, opensuse-leap 16.0, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking (latest)
  • 0.30 Β· fedora 43, fedora 44, ubuntu jammy
, in alpine 3.24, debian bookworm, debian trixie, nixos 26.05, nixpkgs unstable, opensuse-leap 16.0, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

πŸ’– scan_utf8 πŸ’–

okay bestie, welcome to the scan_utf8(3) page, straight out of the Library Functions Manual β€” we're decoding integers from UTF-8 and we're gonna have a good time doing it ✨


πŸŽ€ NAME πŸŽ€

scan_utf8 β€” decode an unsigned integer from UTF-8 encoding

like, that's the whole job. one integer. out of some bytes. slay πŸ’…


🌸 SYNTAX 🌸

#include <libowfat/scan.h>

size_t scan_utf8(const char *src,size_t len,uint32_t *dest);

size_t scan_utf8_sem(const char *src,size_t len,uint32_t *dest);

two functions, same shape, different opinions (we'll get to that πŸ‘€)


✨ DESCRIPTION ✨

so. scan_utf8 decodes an unsigned integer that's been encoded in UTF-8, and it does this from a memory area that's holding binary data. it writes the decoded value into dest and returns the number of bytes it read from src. that's it, that's the whole thing, main character energy fr fr πŸ‘‘

scan_utf8 never reads more than len bytes from src. never ever. boundaries? iconic. 🫢

but okay, here's the part where you gotta pay attention bestie: if the sequence is longer than that, or the memory area contains an invalid sequence, scan_utf8 returns 0 and does not touch dest. it just leaves your variable alone. no notes, no mess, no drama. πŸ’€

fun fact that lives rent free in my head: the length of the longest valid UTF-8 sequence is 6. six. that's the max. πŸ“

now, the tea β˜• β€” scan_utf8 will reject syntactically invalid encodings, but not semantically invalid ones. so it's like "does this look like valid UTF-8 shape-wise? cool, we're good." it's not checking whether the meaning makes sense.

and that's where scan_utf8_sem comes in β€” she's the stricter one πŸ’… scan_utf8_sem will additionally reject surrogates. so if you need the semantic check, that's your girl.


🌈 NOTE 🌈

this is important so like, don't skip it πŸ™ˆ

fmt_utf8 and scan_utf8 implement the encoding from UTF-8, but they're meant to be able to store integers, not just Unicode code points. yes, really. that's the design.

which means: values above 0x10ffff are not valid UTF-8. periodt. 🚫

so if you're using this function to actually parse UTF-8, you need to reject them yourself. (see RFC 3629 β€” it's giving homework but it's giving necessary homework πŸ“š)

lowkey do not forget this step. the function will happily hand you an integer that isn't a legal code point and then you're the one explaining yourself. πŸ’€

πŸ¦‹ SEE ALSO πŸ¦‹

  • fmt_utf8(3) β€” the other half of the duo 🫢
  • scan_utf8_sem(3) β€” the stricter sibling, the one who actually checks for surrogates ✨

and that's the page bestie!! πŸŽ€ go forth and decode responsibly, and remember β€” no cap, reject your values above 0x10ffff 🌈

this was scan_utf8(3) from the Library Functions Manual β€” mwah, catch you on the next man page πŸ’–βœ¨πŸ’