Mail::...h::Util(3pm)

💖 NAME 💖

Mail::SpamAssassin::FuzzyHash::Util — shared utilities for FuzzyHash algorithms ✨

okay bestie so this is the little helper module that all the FuzzyHash algorithms share 🫶 it's giving "we all use the same glue code", and honestly? slay.

🌸 SYNOPSIS 🌸

use Mail::SpamAssassin::FuzzyHash::Util qw(normalize_tokens hamming_distance);

my @tokens = normalize_tokens($text);
my $dist   = hamming_distance($hex_a, $hex_b);

like, that's it. import the two subs, get your tokens, get your distance. main character energy fr fr 💅

🦋 FUNCTIONS 🦋

✨ normalize_tokens($text)

So this one normalises $text and hands you back a list of unique tokens — the kind of tokens that are suitable for fuzzy-hash computation 💖

The way it does that, step by step, no cap:

1. make sure the UTF-8 flag is set ✨
2. apply NFKC normalisation — which folds full-width ASCII spam chars and ligatures together, iconic behaviour honestly 🎀
3. lowercase everything 🍼
4. insert space boundaries around CJK characters 🌈
5. strip anything that is not a Unicode letter, a digit, or a space 💀 (sorry bestie, gone)
6. collapse whitespace 🧚
7. split and deduplicate 👑

and if a text produces literally no tokens after all that normalisation? you get ('__empty__') back. she's empty, but she's still returned 🥲

✨ hamming_distance($hex_a, $hex_b)

Returns the Hamming distance between two equal-length lowercase hex strings — that's the number of differing bits between them 🔥

heads up babe: the strings must have even length. that's not a vibe, that's a requirement 💅

written by the fine folks behind User Contributed Perl Documentation — this page is <code>Mail::...h::Util(3pm)</code>, which is the cute little shortened name for Mail::SpamAssassin::FuzzyHash::Util 🌸

perl v5.40.1 · 2026-07-23 🎀 that's a wrap, ily, go normalise some tokens ✨