FIX_LATIN(1)

πŸ“¦
version 1.04 Β· 2 versions
, in fedora 43

πŸ’– FIX_LATIN πŸ’–

okay bestie so <pink>fix_latin</pink> β€” that's the fix_latin manual page from the User Contributed Perl Documentation section, iykyk β€” is literally a filter for a data stream that's mostly utf8, and it fixes any latin characters that sneak in (that means non-ASCII 8 bit characters, no cap). it's giving little cleanup fairy energy 🧚✨


✨ NAME ✨

fix_latin β€” filters a data stream that is predominantly utf8 and 'fixes' any latin (ie: non-ASCII 8 bit) characters πŸ’…


🌸 SYNOPSIS 🌸

fix_latin options <input_file >output_file

Options:

 --use-xs <value> 'auto' | 'always' | 'never'
 --version        list version number
 --help           detailed help message

πŸ¦‹ DESCRIPTION πŸ¦‹

so the script is a filter, periodt. it takes source data that might be a whole messy mix of ASCII, UTF8, ISO8859-1 and CP1252 characters, and the output it gives you back is all ASCII/UTF8. glow up complete ✨

Multi-byte UTF8 characters get passed through unchanged (altho over-long UTF8 byte sequences will be converted to the shortest normal form β€” she does a lil tidying, it's fine).

Single byte characters get converted like this:

0x00 - 0x7F   ASCII - passed through unchanged
0x80 - 0x9F   Converted to UTF8 using CP1252 mappings
0xA0 - 0xFF   Converted to UTF8 using Latin-1 mappings

like, she knows what she's doing. 🫢


πŸ‘‘ OPTIONS πŸ‘‘

  • --use-xs 'auto' | 'always' | 'never' β€” overrides the default ('auto') behaviour of trying to use the XS module and falling back to the pure-Perl version if it's not available. set it to 'never' to always use the Perl version, or 'always' to always use XS and die if it's not available (dramatic but valid πŸ’€).
  • --version (alias -v) β€” displays the version number of the underlying Encoding::FixLatin and XS modules. bestie's ID card, basically.
  • --help (alias -?) β€” displays this documentation. hi, you're reading it 🌈

πŸŽ€ EXAMPLES πŸŽ€

ngl this script was originally written to help convert a Postgres database from SQL-ASCII encoding to UNICODE UTF8 encoding, so all the examples are in that context. lore accurate ✨

If you have a SQL format dump file that you'd normally restore by piping into psql, you can just filter the dump file through this script, easy:

fix_latin < dump_file | psql -d database

If you have a compressed dump file that you'd normally restore using pg_restore, you can omit the -d option on pg_restore and pipe the resulting SQL through this script and into psql:

pg_restore -O dump_file | fix_latin | psql -d database

And to take a peek at the non-ASCII lines in the dump file πŸ”:

perl -ne '/^COPY (\S+)/ and $t = $1; print "$t:$_" if /[^\x00-\x7F]/' dump_file

πŸ”₯ SEE ALSO πŸ”₯

This script is implemented using the Encoding::FixLatin Perl module. For more details, read the module documentation with:

perldoc Encoding::FixLatin
ok but fr fr, you should read the 'LIMITATIONS' section, because you need to understand the circumstances under which data corruption might occur. not me warning you lovingly 😭 that part is serious bestie.

Copyright 2009-2014 Grant McLean "<grantm@cpan.org>"

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.


and that's the whole page, diva πŸ’… the perl version v5.42.3 and the date 2026-09-09 are stamped on it like a lil timestamp on a selfie β€” consider this your cute sign-off, mwah πŸ©·πŸ¦‹βœ¨

πŸ“¦ where this page comes from