PERLUNICOOK(1perl)
💖 perlunicook(1perl) — the Perl Programmers Reference Guide 💖
okay bestie, pull up a chair, grab a boba 🧋, because this is the Unicode cookbook and it's about to be your roman empire.
✨ NAME ✨
perlunicook — cookbookish examples of handling Unicode in Perl 🎀
🌸 DESCRIPTION 🌸
Sit down, bestie, because this manpage is a whole recipe box:</purgle> it's full of short recipes showing you how to pull off the common Unicode operations in Perl, plus one complete program at the very end (iykyk, we'll get there 👑).
little housekeeping note 🫶 — any undeclared variables hanging out in the individual recipes are just assumed to already have some appropriate value in them from earlier. No mystery variables, no jump scares.
🦋 EXAMPLES 🦋
the moment you've all been waiting for, let's get into the recipes 💅
🍓 ℞ 0: Standard preamble
Lowkey the most important one, no cap. Unless it says otherwise, every single example below needs this standard preamble to actually work right — and yes, you gotta tweak the #! to match your system.
#!/usr/bin/env perl
use v5.36; # or later to get "unicode_strings" feature,
# plus strict, warnings
use utf8; # so literals and identifiers can be in UTF-8
use warnings qw(FATAL utf8); # fatalize encoding glitches
use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
use charnames qw(:full :short); # unneeded in v5.16Fair warning, bestie: this does make even Unix programmers binmode their binary streams, or open them with ":raw" — but honestly? that's the only way to get at them portably anyway. Periodt. 💅
WARNING:use autodie(pre 2.26) anduse opendo not get along with each other. They are beefing. Do not invite them to the same party. 😭
💜 ℞ 1: Generic Unicode-savvy filter
the vibe here is simple: always decompose on the way in, then recompose on the way out. iconic behaviour, honestly ✨
use Unicode::Normalize;
while (<>) {
$_ = NFD($_); # decompose + reorder canonically
...
} continue {
print NFC($_); # recompose (where possible) + reorder canonically
}🧚 ℞ 2: Fine-tuning Unicode warnings
As of v5.14, Perl is not playing — it distinguishes three subclasses of UTF-8 warnings. Turn off the ones you don't care about 💅
use v5.14; # subwarnings unavailable any earlier
no warnings "nonchar"; # the 66 forbidden non-characters
no warnings "surrogate"; # UTF-16/CESU-8 nonsense
no warnings "non_unicode"; # for codepoints over 0x10_FFFF🎀 ℞ 3: Declare source in utf8 for identifiers and literals
Okay so this is the all-critical one: without that use utf8 declaration, putting UTF-8 in your literals and identifiers will not work right. If you used the standard preamble above, congrats bestie, you already did it 🎉 And if you did, you can pull off stuff like this:
use utf8;
my $measure = "Ångström";
my @μsoft = qw( cp852 cp1251 cp1252 );
my @ὑπέρμεγας = qw( ὑπέρ μεγας );
my @鯉 = qw( koi8-f koi8-u koi8-r );
my $motto = "👪 💗 🐪"; # FAMILY, GROWING HEART, DROMEDARY CAMEL😭 If you forget use utf8, high bytes will get misunderstood as separate characters, and nothing will work right. Not even a little bit. It's giving chaos.
⭐ ℞ 4: Characters and their numbers
The ord and chr functions work transparently on ALL codepoints — not just ASCII alone, and honestly not even just Unicode alone. Main character energy tbh 👑
# ASCII characters
ord("A")
chr(65)
# characters from the Basic Multilingual Plane
ord("Σ")
chr(0x3A3)
# beyond the BMP
ord("𝑛") # MATHEMATICAL ITALIC SMALL N
chr(0x1D45B)
# beyond Unicode! (up to MAXINT)
ord("\x{20_0000}")
chr(0x20_0000)🌈 ℞ 5: Unicode literals by character number
In an interpolated literal — whether that's a double-quoted string or a regex — you can specify a character by its number using the "\x{HHHHHH}" escape. Chef's kiss. 👨🍳💋
String: "\x{3a3}"
Regex: /\x{3a3}/
String: "\x{1d45b}"
Regex: /\x{1d45b}/
# even non-BMP ranges in regex work fine
/[\x{1D434}-\x{1D467}]/🫶 ℞ 6: Get character name by number
use charnames ();
my $name = charnames::viacode(0x03A3);💅 ℞ 7: Get character number by name
use charnames ();
my $number = charnames::vianame("GREEK CAPITAL LETTER SIGMA");🦋 ℞ 8: Unicode named characters
Use the "\N{charname}" notation to grab the character by that name for use in interpolated literals (double-quoted strings and regexes). In v5.16, there's an implicit
use charnames qw(:full :short);But prior to v5.16, bestie, you gotta be explicit about which set of charnames you want. The ":full" names are the official Unicode character name, alias, or sequence — and they all share a namespace.
use charnames qw(:full :short latin greek);
"\N{MATHEMATICAL ITALIC SMALL N}" # :full
"\N{GREEK CAPITAL LETTER SIGMA}" # :fullAnything else is a Perl-specific convenience abbreviation. Specify one or more scripts by name if you want short names that are script-specific:
"\N{Greek:Sigma}" # :short
"\N{ae}" # latin
"\N{epsilon}" # greekThe v5.16 release also supports a ":loose" import for loose matching of character names, which works exactly like loose matching of property names: it disregards case, whitespace, and underscores 💅
"\N{euro sign}" # :loose (from v5.16)Starting in v5.32, you can also use
qr/\p{name=euro sign}/to get official Unicode named characters in regular expressions. Loose matching is always done for these. No cap. ✨
🐪 ℞ 9: Unicode named sequences
These look just like character names but return MULTIPLE codepoints 👀 Notice the %vx vector-print functionality in printf. Cute, right?
use charnames qw(:full);
my $seq = "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}";
printf "U+%v04X\n", $seq;
U+0100.0300🍓 ℞ 10: Custom named characters
Use ":alias" to give your own lexically scoped nicknames to existing characters — or even to give unnamed private-use characters useful names. Giving characters a glow-up, honestly. 💅
use charnames ":full", ":alias" => {
ecute => "LATIN SMALL LETTER E WITH ACUTE",
"APPLE LOGO" => 0xF8FF, # private use character
};
"\N{ecute}"
"\N{APPLE LOGO}"👑 ℞ 11: Names of CJK codepoints
Sinograms like "東京" come back with character names of CJK UNIFIED IDEOGRAPH-6771 and CJK UNIFIED IDEOGRAPH-4EAC, because their "names" vary. The CPAN Unicode::Unihan module has a large database for decoding these (and a whole lot more) — provided you know how to understand its output 🙈
# cpan -i Unicode::Unihan
use Unicode::Unihan;
my $str = "東京";
my $unhan = Unicode::Unihan->new;
for my $lang (qw(Mandarin Cantonese Korean JapaneseOn JapaneseKun)) {
printf "CJK $str in %-12s is ", $lang;
say $unhan->$lang($str);
}prints:
CJK 東京 in Mandarin is DONG1JING1
CJK 東京 in Cantonese is dung1ging1
CJK 東京 in Korean is TONGKYENG
CJK 東京 in JapaneseOn is TOUKYOU KEI KIN
CJK 東京 in JapaneseKun is HIGASHI AZUMAMIYAKOIf you have a specific romanization scheme in mind, use the specific module:
# cpan -i Lingua::JA::Romanize::Japanese
use Lingua::JA::Romanize::Japanese;
my $k2r = Lingua::JA::Romanize::Japanese->new;
my $str = "東京";
say "Japanese for $str is ", $k2r->chars($str);prints
Japanese for 東京 is toukyouSlay. 🔥
🌸 ℞ 12: Explicit encode/decode
On rare occasion — like, say, a database read — you may be handed encoded text that you need to decode. Rude of it, but okay.
use Encode qw(encode decode);
my $chars = decode("shiftjis", $bytes, 1);
# OR
my $bytes = encode("MIME-Header-ISO_2022_JP", $chars, 1);For streams all in the same encoding, DON'T use encode/decode. Instead set the file encoding when you open the file, or immediately after, with binmode as described later below. 🫶
💜 ℞ 13: Decode program arguments as utf8
$ perl -CA ...
or
$ export PERL_UNICODE=A
or
use Encode qw(decode);
@ARGV = map { decode('UTF-8', $_, 1) } @ARGV;🧚 ℞ 14: Decode program arguments as locale encoding
# cpan -i Encode::Locale
use Encode qw(locale);
use Encode::Locale;
# use "locale" as an arg to encode/decode
@ARGV = map { decode(locale => $_, 1) } @ARGV;🎀 ℞ 15: Declare STD{IN,OUT,ERR} to be utf8
Pick your fighter, bestie: use a command-line option, an environment variable, or else call binmode explicitly 💅
$ perl -CS ...
or
$ export PERL_UNICODE=S
or
use open qw(:std :encoding(UTF-8));
or
binmode(STDIN, ":encoding(UTF-8)");
binmode(STDOUT, ":utf8");
binmode(STDERR, ":utf8");⭐ ℞ 16: Declare STD{IN,OUT,ERR} to be in locale encoding
# cpan -i Encode::Locale
use Encode;
use Encode::Locale;
# or as a stream for binmode or open
binmode STDIN, ":encoding(console_in)" if -t STDIN;
binmode STDOUT, ":encoding(console_out)" if -t STDOUT;
binmode STDERR, ":encoding(console_out)" if -t STDERR;🌈 ℞ 17: Make file I/O default to utf8
Files opened without an encoding argument will be in UTF-8: ✨
$ perl -CD ...
or
$ export PERL_UNICODE=D
or
use open qw(:encoding(UTF-8));🫶 ℞ 18: Make all I/O and args default to utf8
$ perl -CSDA ...
or
$ export PERL_UNICODE=SDA
or
use open qw(:std :encoding(UTF-8));
use Encode qw(decode);
@ARGV = map { decode('UTF-8', $_, 1) } @ARGV;the whole package deal, honestly. 🔥
💅 ℞ 19: Open file with specific encoding
Specify stream encoding. THIS is the normal way to deal with encoded text — not by calling low-level functions. Okay? Okay. 💖
# input file
open(my $in_file, "< :encoding(UTF-16)", "wintext");
OR
open(my $in_file, "<", "wintext");
binmode($in_file, ":encoding(UTF-16)");
THEN
my $line = <$in_file>;
# output file
open($out_file, "> :encoding(cp1252)", "wintext");
OR
open(my $out_file, ">", "wintext");
binmode($out_file, ":encoding(cp1252)");
THEN
print $out_file "some text\n";More layers than just the encoding can be specified here. For example, the incantation ":raw :encoding(UTF-16LE) :crlf" includes implicit CRLF handling. Layers on layers, we love a stacked moment. ✨
🦋 ℞ 20: Unicode casing
Unicode casing is VERY different from ASCII casing. Buckle up 💀
uc("henry ⅷ") # "HENRY Ⅷ"
uc("tschüß") # "TSCHÜSS" notice ß => SS
# both are true:
"tschüß" =~ /TSCHÜSS/i # notice ß => SS
"Σίσυφος" =~ /ΣΊΣΥΦΟΣ/i # notice Σ,σ,ς samenessthe Σ, σ, ς sameness is genuinely so iconic of them. 👑
🍓 ℞ 21: Unicode case-insensitive comparisons
Also available in the CPAN Unicode::CaseFold module, the new fc "foldcase" function from v5.16 grants access to the same Unicode casefolding as the /i pattern modifier has always used:
use feature "fc"; # fc() function is from v5.16
# sort case-insensitively
my @sorted = sort { fc($a) cmp fc($b) } @list;
# both are true:
fc("tschüß") eq fc("TSCHÜSS")
fc("Σίσυφος") eq fc("ΣΊΣΥΦΟΣ")the way that both of those are just true. obsessed. 💅
🐪 ℞ 22: Match Unicode linebreak sequence in regex
A Unicode linebreak matches the two-character CRLF grapheme — or any of seven vertical whitespace characters. Super good for dealing with textfiles coming from different operating systems 🫶 (we've all been there, bestie)
\R
s/\R/\n/g; # normalize all linebreaks to \nnormalize all the linebreaks, slay. ✨
℞ 23: Get character category 🔍
okay so if you wanna know the general category of a numeric codepoint — like, is it a letter, a punctuation, what's the vibe — this is the girl for the job.
use Unicode::UCD qw(charinfo);
my $cat = charinfo(0x3A3)->{category}; # "Lu"omg "Lu" means Letter-uppercase, iykyk 👑
℞ 24: Disabling Unicode-awareness in builtin charclasses 🙈
bestie, sometimes you just wanna turn OFF the Unicode brain of \w, \b, \s, \d, and the POSIX classes. you can do it for this whole scope, or for just one regex. girl your choice 💅
use v5.14;
use re "/a";
# OR
my($num) = $str =~ /(\d+)/a;or you can use those specific un-Unicode properties instead, like \p{ahex} and \p{POSIX_Digit}. no cap, properties still work totally normally no matter what charset modifiers (/d /u /l /a /aa) are in effect ✨
℞ 25: Match Unicode properties in regex with \p, \P 🎀
these all match a single codepoint that has the given property. swap in \P instead of \p to match one codepoint lacking that property 💀
\pL, \pN, \pS, \pP, \pM, \pZ, \pC
\p{Sk}, \p{Ps}, \p{Lt}
\p{alpha}, \p{upper}, \p{lower}
\p{Latin}, \p{Greek}
\p{script_extensions=Latin}, \p{scx=Greek}
\p{East_Asian_Width=Wide}, \p{EA=W}
\p{Line_Break=Hyphen}, \p{LB=HY}
\p{Numeric_Value=4}, \p{NV=4}℞ 26: Custom character properties 🌈
you can define your own custom character properties at compile-time, for use in regexes. main character energy fr fr
# using private-use characters
sub In_Tengwar { "E000\tE07F\n" }
if (/\p{In_Tengwar}/) { ... }# blending existing properties
sub Is_GraecoRoman_Title {<<'END_OF_SET'}
+utf8::IsLatin
+utf8::IsGreek
&utf8::IsTitle
END_OF_SET
if (/\p{Is_GraecoRoman_Title}/) { ... }℞ 27: Unicode normalization 🦋
the standard move: render into NFD on input and NFC on output. and if you use NFKC or NFKD, that improves recall on searches — assuming you've already done the same thing to the text you're searching 🫶
lowkey important: this is about wayyy more than just pre-combined compatibility glyphs. it also reorders marks according to their canonical combining classes and weeds out singletons. iconic.
use Unicode::Normalize;
my $nfd = NFD($orig);
my $nfc = NFC($orig);
my $nfkd = NFKD($orig);
my $nfkc = NFKC($orig);℞ 28: Convert non-ASCII Unicode numerics 🔢
unless you've used /a or /aa, \d matches more than just ASCII digits. but here's the tea: Perl's implicit string-to-number conversion does not currently recognize these. so here's how you convert such strings manually 💅
use utf8;
use v5.14; # needed for num() function
use Unicode::UCD qw(num);
my $str = "got Ⅻ and ४५६७ and ⅞ and 兆 here";
my @nums = ();
while ($str =~ /(\d+|\N)/g) { # not just ASCII!
push @nums, num($1);
}
say "@nums"; # 12 4567 0.875 1000000000000
use charnames qw(:full);
my $nv = num("\N{VAI DIGIT ONE}\N{VAI DIGIT TWO}");
say $nv; # 12℞ 29: Match Unicode grapheme cluster in regex ✨
so the "characters" a programmer sees are codepoints, matched by /./s. but the "characters" a user sees are graphemes, matched by /\X/. two totally different girls 💖
# Find vowel *plus* any combining diacritics, underlining, etc.
my $nfd = NFD($orig);
$nfd =~ / (?=[aeiou]) \X /xi℞ 30: Extract by grapheme instead of by codepoint (regex) 🍓
# match and grab five first graphemes
my($first_five) = $str =~ /^ ( \X{5} ) /x;℞ 31: Extract by grapheme instead of by codepoint (substr) 🌸
# cpan -i Unicode::GCString
use Unicode::GCString;
my $gcs = Unicode::GCString->new($str);
my $first_five = $gcs->substr(0, 5);℞ 32: Reverse string by grapheme 💀
okay this one is a tragedy if you do it wrong. reversing by codepoint messes up diacritics — it'll turn "crème brûlée" into "éelurb emerc" instead of into "eélûrb emèrc". so reverse by grapheme instead, bestie.
both of these approaches work right no matter what normalization the string is in 🫶
$str = join("", reverse $str =~ /\X/g);
# OR: cpan -i Unicode::GCString
use Unicode::GCString;
$str = reverse Unicode::GCString->new($str);℞ 33: String length in graphemes 📏
the string "brûlée" has six graphemes but up to eight codepoints. so this counts by grapheme, not by codepoint:
my $str = "brûlée";
my $count = 0;
while ($str =~ /\X/g) { $count++ }
# OR: cpan -i Unicode::GCString
use Unicode::GCString;
my $gcs = Unicode::GCString->new($str);
my $count = $gcs->length;℞ 34: Unicode column-width for printing 🖨️✨
Perl's printf, sprintf, and format think all codepoints take up 1 print column — but many take 0 or 2. rude, honestly. here, to show that normalization makes no difference, we print out both forms:
use Unicode::GCString;
use Unicode::Normalize;
my @words = qw/crème brûlée/;
@words = map { NFC($_), NFD($_) } @words;
for my $str (@words) {
my $gcs = Unicode::GCString->new($str);
my $cols = $gcs->columns;
my $pad = " " x (10 - $cols);
say $str, $pad, " |";
}and it generates this, to show that it pads correctly no matter the normalization 🎀
crème |
creme |
brûlée |
brulee |℞ 35: Unicode collation 👑
text sorted by numeric codepoint follows no reasonable alphabetic order, ok? it's giving chaos. so use the UCA for sorting text 💅
use Unicode::Collate;
my $col = Unicode::Collate->new();
my @list = $col->sort(@old_list);peep the ucsort program from the Unicode::Tussle CPAN module for a convenient command-line interface to this module 👀
℞ 36: Case- and accent-insensitive Unicode sort 🌈
specify a collation strength of level 1 to ignore case and diacritics, so it's only looking at the basic character. simple as ✨
use Unicode::Collate;
my $col = Unicode::Collate->new(level => 1);
my @list = $col->sort(@old_list);℞ 37: Unicode locale collation 🌍
some locales have their own special sorting rules, and they deserve respect 🫶
# either use v5.12, OR: cpan -i Unicode::Collate::Locale
use Unicode::Collate::Locale;
my $col = Unicode::Collate::Locale->new(locale => "de__phonebook");
my @list = $col->sort(@old_list);and btw the ucsort program mentioned above accepts a "--locale" parameter 💅
℞ 38: Making cmp work on text instead of codepoints 😤
instead of this:
@srecs = sort {
$b->{AGE} <=> $a->{AGE}
||
$a->{NAME} cmp $b->{NAME}
} @recs;use this:
my $coll = Unicode::Collate->new();
for my $rec (@recs) {
$rec->{NAME_key} = $coll->getSortKey( $rec->{NAME} );
}
@srecs = sort {
$b->{AGE} <=> $a->{AGE}
||
$a->{NAME_key} cmp $b->{NAME_key}
} @recs;℞ 39: Case- and accent-insensitive comparisons 💖
use a collator object to compare Unicode text by character instead of by codepoint. slay.
use Unicode::Collate;
my $es = Unicode::Collate->new(
level => 1,
normalization => undef
);
# now both are true:
$es->eq("García", "GARCIA" );
$es->eq("Márquez", "MARQUEZ");℞ 40: Case- and accent-insensitive locale comparisons 🌸
same thing, but in a specific locale:
my $de = Unicode::Collate::Locale->new(
locale => "de__phonebook",
);
# now this is true:
$de->eq("tschüß", "TSCHUESS"); # notice ü => UE, ß => SS℞ 41: Unicode linebreaking ✂️
break up text into lines according to Unicode rules. iconic.
# cpan -i Unicode::LineBreak
use Unicode::LineBreak;
use charnames qw(:full);
my $para = "This is a super\N{HYPHEN}long string. " x 20;
my $fmt = Unicode::LineBreak->new;
print $fmt->break($para), "\n";℞ 42: Unicode text in DBM hashes, the tedious way 😮💨
using a regular Perl string as a key or value for a DBM hash will trigger a wide character exception if any codepoints won't fit into a byte 😭 so here's how to manually manage the translation:
use DB_File;
use Encode qw(encode decode);
tie %dbhash, "DB_File", "pathname";
# STORE
# assume $uni_key and $uni_value are abstract Unicode strings
my $enc_key = encode("UTF-8", $uni_key, 1);
my $enc_value = encode("UTF-8", $uni_value, 1);
$dbhash{$enc_key} = $enc_value;
# FETCH
# assume $uni_key holds a normal Perl string (abstract Unicode)
my $enc_key = encode("UTF-8", $uni_key, 1);
my $enc_value = $dbhash{$enc_key};
my $uni_value = decode("UTF-8", $enc_value, 1);℞ 43: Unicode text in DBM hashes, the easy way 🧚
here's how to implicitly manage the translation — all encoding and decoding is done automatically, just like with streams that have a particular encoding attached to them. no manual labour required 💅
use DB_File;
use DBM_Filter;
my $dbobj = tie %dbhash, "DB_File", "pathname";
$dbobj->Filter_Value("utf8"); # this is the magic bit
# STORE
# assume $uni_key and $uni_value are abstract Unicode strings
$dbhash{$uni_key} = $uni_value;
# FETCH
# $uni_key holds a normal Perl string (abstract Unicode)
my $uni_value = $dbhash{$uni_key};℞ 44: PROGRAM: Demo of Unicode collation and printing 👑💖
here's a full program showing how to make use of locale-sensitive sorting, Unicode casing, and managing print widths when some of the characters take up zero or two columns, not just one column each time. and when you run it?? the following program produces this gorgeously aligned output 😭✨
Crème Brûlée....... €2.00
Éclair............. €1.60
Fideuà............. €4.20
Hamburger.......... €6.00
Jamón Serrano...... €4.45
Linguiça........... €7.00
Pâté............... €4.15
Pears.............. €2.00
Pêches............. €2.25
Smørbrød........... €5.75
Spätzle............ €5.50
Xoriço............. €3.00
Γύρος.............. €6.50
막걸리............. €4.00
おもち............. €2.65
お好み焼き......... €8.00
シュークリーム..... €1.85
寿司............... €9.99
包子............... €7.50here's that program.
#!/usr/bin/env perl
# umenu - demo sorting and printing of Unicode food
#
# (obligatory and increasingly long preamble)
#
use v5.36;
use utf8;
use warnings qw(FATAL utf8); # fatalize encoding faults
use open qw(:std :encoding(UTF-8)); # undeclared streams in UTF-8
use charnames qw(:full :short); # unneeded in v5.16
# std modules
use Unicode::Normalize; # std perl distro as of v5.8
use List::Util qw(max); # std perl distro as of v5.10
use Unicode::Collate::Locale; # std perl distro as of v5.14
# cpan modules
use Unicode::GCString; # from CPAN my %price = (
"γύρος" => 6.50, # gyros
"pears" => 2.00, # like um, pears
"linguiça" => 7.00, # spicy sausage, Portuguese
"xoriço" => 3.00, # chorizo sausage, Catalan
"hamburger" => 6.00, # burgermeister meisterburger
"éclair" => 1.60, # dessert, French
"smørbrød" => 5.75, # sandwiches, Norwegian
"spätzle" => 5.50, # Bayerisch noodles, little sparrows
"包子" => 7.50, # bao1 zi5, steamed pork buns, Mandarin
"jamón serrano" => 4.45, # country ham, Spanish
"pêches" => 2.25, # peaches, French
"シュークリーム" => 1.85, # cream-filled pastry like eclair
"막걸리" => 4.00, # makgeolli, Korean rice wine
"寿司" => 9.99, # sushi, Japanese
"おもち" => 2.65, # omochi, rice cakes, Japanese
"crème brûlée" => 2.00, # crema catalana
"fideuà" => 4.20, # more noodles, Valencian
# (Catalan=fideuada)
"pâté" => 4.15, # goose liver paste, French
"お好み焼き" => 8.00, # okonomiyaki, Japanese
);
my $width = 5 + max map { colwidth($_) } keys %price;
# So the Asian stuff comes out in an order that someone
# who reads those scripts won't freak out over; the
# CJK stuff will be in JIS X 0208 order that way.
my $coll = Unicode::Collate::Locale->new(locale => "ja");
for my $item ($coll->sort(keys %price)) {
print pad(entitle($item), $width, ".");
printf " €%.2f\n", $price{$item};
}
sub pad ($str, $width, $padchar) {
return $str . ($padchar x ($width - colwidth($str)));
}
sub colwidth ($str) {
return Unicode::GCString->new($str)->columns;
}
sub entitle ($str) {
$str =~ s{ (?=\pL)(\S) (\S*) }
{ ucfirst($1) . lc($2) }xge;
return $str;
}🌈 SEE ALSO 🌈
bestie there's a whole squad of manpages and CPAN modules that vibe with this one, go read them:
perlunicodeperlunipropsperlreperlrecharclassperluniintroperlunitutperlunifaqPerlIODB_FileDBM_FilterDBM_Filter::utf8EncodeEncode::LocaleUnicode::UCDUnicode::NormalizeUnicode::GCStringUnicode::LineBreakUnicode::CollateUnicode::Collate::LocaleUnicode::UnihanUnicode::CaseFoldUnicode::TussleLingua::JA::Romanize::JapaneseLingua::ZH::Romanize::PinyinLingua::KO::Romanize::Hangul
okay and the Unicode::Tussle CPAN module?? literally it ships a whole arsenal of programs for doing Unicode things, including these ones that fully or partly replace your standard utilities 👑
tcgrepinstead ofegrepuniquoteinstead ofcat -vorhexdumpuniwcinstead ofwcunilookinstead oflookunifmtinstead offmtucsortinstead ofsort
and for exploring Unicode character names and character properties, check out its uniprops, unichars, and uninames programs. ✨ it also supplies these, all of which are general filters that do Unicode-y things:
unititleandunicapsuniwideanduninarrowunisupersandunisubsnfd,nfc,nfkd, andnfkcuc,lc, andtc
finally finally, go look at the published Unicode Standard 💖 (the page numbers are from version 6.0.0), including these specific annexes and technical reports:
- §3.13 Default Case Algorithms, page 113; §4.2 Case, pages 120–122; Case Mappings, page 166–172, especially Caseless Matching starting on page 170.
- UAX #44: Unicode Character Database
- UTS #18: Unicode Regular Expressions
- UAX #15: Unicode Normalization Forms
- UTS #10: Unicode Collation Algorithm
- UAX #29: Unicode Text Segmentation
- UAX #14: Unicode Line Breaking Algorithm
- UAX #11: East Asian Width
👑 AUTHOR 👑
Tom Christiansen <tchrist@perl.com> wrote this, with occasional kibbitzing from Larry Wall and Jeffrey Friedl in the background. the group project energy is unmatched fr fr 🫶
🎀 COPYRIGHT AND LICENCE 🎀
Copyright © 2012 Tom Christiansen.
This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself. slay, do what you want 💅
Most of these examples taken from the current edition of the “Camel Book”; that is, from the 4ᵗʰ Edition of Programming Perl, Copyright © 2012 Tom Christiansen <et al.>, 2012-02-13 by O’Reilly Media. The code itself is freely redistributable, and you are encouraged to transplant, fold, spindle, and mutilate any of the examples in this manpage however you please for inclusion into your own programs without any encumbrance whatsoever. Acknowledgement via code comment is polite but not required. ✨
no cap, you can straight up take this code, fold it, spindle it, mutilate it, put it in your own programs, zero strings attached. a lil code comment saying thanks is cute but nobody's gonna come for you if you skip it 💖
🦋 REVISION HISTORY 🦋
- v1.0.0 – first public release, 2012-02-27
and that's a wrap bestie!! 🌸 this whole thing is from perl v5.44.0 and it's dated 2026-10-17 — go forth and cook with Unicode, you're gonna eat 🔥💜
📦 where this page comes from
- arch rolling: perl 5.44.0-1 · Artistic-1.0-Perl AND GPL-1.0-or-later
- 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms