LucyX:...eldSim(3pm)

💖 NAME 💖

LucyX::Index::LongFieldSim — a Similarity that's optimized for long fields, no cap. ✨

---

🌸 SYNOPSIS 🌸

package MySchema::body;
use base qw( Lucy::Plan::FullTextType );
use LucyX::Index::LongFieldSim;
sub make_similarity { LucyX::Index::LongFieldSim->new }

---

🦋 DESCRIPTION 🦋

okay so bestie, here's the tea: Apache Lucy's default Similarity implementation has a whole bias toward extremely short fields. It's giving favoritism. 😭

Lucy::Index::Similarity

| more weight
| *
|  **
|    ***
|       **********
|                 ********************
|                                     *******************************
| less weight                                                        ****
|------------------------------------------------------------------------
  fewer tokens                                              more tokens

And LongFieldSim? It eliminates that bias entirely. Iconic behavior, honestly. 👑

LucyX::Index::LongFieldSim

| more weight
|
|
|
|*****************
|                 ********************
|                                     *******************************
| less weight                                                        ****
|------------------------------------------------------------------------
  fewer tokens                                              more tokens

Now listen — in most cases, that default bias toward short fields is actually the moment. Desirable, even. Say you've got two documents:

• "George Washington"
• "George Washington Carver"

If a user searches for "george washington", you want the exact title match to show up first. And under the default Similarity implementation it will, because the "Carver" in "George Washington Carver" dilutes the impact of the other two tokens. Balance restored. 💅

<b>However</b>, under LongFieldSim, those two titles will yield equal scores. In this particular case that would be bad, but lowkey it could be good in another. Watch this:

"George Washington Carver is cool."

"George Washington Carver was born on the eve of the US Civil War, in 1864. His exact date of birth is unknown... Carver's research in crop rotation revolutionized agriculture..."

The first document is succinct, sure, but it's also useless. And unfortunately, the default similarity will assess it as extremely relevant to a query of "george washington carver". 🫠

But under LongFieldSim, the short-field bias is eliminated, and the extra mentions of Carver's name in the second document add up to a higher score and a higher rank. The way that long doc ate and left no crumbs... fr fr. 🔥

---

and that's the whole page, bestie! 🎀 This lil' gem comes to you from the User Contributed Perl Documentation, straight outta perl v5.42.0, dated 2025-07-25. go forth and rank responsibly 💜🦋