Lingua::Stem::En(3pm)

📦
version 2.31 · 6 versions
, in opensuse-leap 16.0

💖 Lingua::Stem::En(3pm) — Porter's stemming algorithm for 'generic' English 💖

okay bestie, welcome to the Lingua::Stem::En(3pm) page from the User Contributed Perl Documentation. we're doing Porter stemming for 'generic' English, and it's giving very specific algorithm energy. 💅

✨ NAME ✨

Lingua::Stem::En — Porter's stemming algorithm for 'generic' English. periodt.

🌸 SYNOPSIS 🌸

           use Lingua::Stem::En;
           my $stems   = Lingua::Stem::En::stem({ -words => $word_list_reference,
                                               -locale => 'en',
                                           -exceptions => $exceptions_hash,
                                            });

💖 DESCRIPTION 💖

okay so this routine takes its parameters and applies the Porter Stemming Algorithm to them, then returns the stemmed words. that's the whole job. 💖

it's derived from the C program stemmer.c as found in freewais and elsewhere, which contains these notes:

Purpose:    Implementation of the Porter stemming algorithm documented
            in: Porter, M.F., "An Algorithm For Suffix Stripping,"
            Program 14 (3), July 1980, pp. 130-137.
Provenance: Written by B. Frakes and C. Cox, 1986.

I have re-interpreted the areas that use Frakes and Cox's WordSize function. fair warning: my version may misbehave on short words starting with y, but honestly I can't think of any examples. 😭

the step numbers correspond to Frakes and Cox, and they're probably in Porter's article (which I've not seen, ngl). Porter's algorithm still has rough spots — like current/currency and -ings words — and I've not attempted to cure them. but I did add support for the British -ise suffix, so there's that. ✨

🔥 CHANGES 🔥

bestie, the lore drops:

  • 1999.06.15 — Changed to '.pm' module, moved into Lingua::Stem namespace, optionalized the export of the 'stem' routine into the caller's namespace, added named parameters.
  • 1999.06.24 — Switch core implementation of the Porter stemmer to the one written by Jim Richardson <jimr@maths.usyd.edu.au>.
  • 2000.08.25 — 2.11 Added stemming cache.
  • 2000.09.14 — 2.12 Fixed major :( implementation error of Porter's algorithm. Error was entirely my fault — I completely forgot to include rule sets 2,3, and 4 starting with Lingua::Stem 0.30. — Jerilyn Franz.
  • 2003.09.28 — 2.13 Corrected documentation error pointed out by Simon Cozens.
  • 2005.11.20 — 2.14 Changed rule declarations to conform to Perl style convention for 'private' subroutines. Changed Exporter invokation to more portable 'require' vice 'use'.
  • 2006.02.14 — 2.15 Added ability to pass word list by 'handle' for in-place stemming.
  • 2009.07.27 — 2.16 Documentation Fix.
  • 2020.06.20 — 2.30 Version renumber for module consistency.
  • 2020.09.26 — 2.31 Fix for Latin1/UTF8 issue in documentation.

🧚 METHODS 🧚

💅 stem

stem({ -words => \@words, -locale => 'en', -exceptions => \%exceptions
});

this stems a list of passed words using the rules of US English, and returns an anonymous array reference to the stemmed words. yes bestie, an array ref. 💖

example time:

             my @words         = ( 'wordy', 'another' );
             my $stemmed_words = Lingua::Stem::En::stem({ -words => \@words,
                                                         -locale => 'en',
                                                     -exceptions => \%exceptions,
                                     });

if the first element of @words is a list reference, then the stemming is performed 'in place' on that list — it modifies the passed list directly instead of copying it to a new array. lowkey efficient.

this is only useful if you do not need to keep the original list. if you do need to keep the original list, use the normal semantic of having 'stem' return a new list instead — that is faster than making your own copy and using the 'in place' semantics, since the primary difference between 'in place' and 'by value' stemming is the creation of a copy of the original list. if you don't need the original list, then the 'in place' stemming is about 60% faster. slay. 🔥

example of 'in place' stemming:

             my $words         = [ 'wordy', 'another' ];
             my $stemmed_words = Lingua::Stem::En::stem({ -words => [$words],
                                     -locale => 'en',
                                 -exceptions => \%exceptions,
                                 });

the 'in place' mode returns a reference to the original list with the words stemmed. periodt.

🎀 stem_caching

stem_caching({ -level => 0|1|2 });

sets the level of stem caching. choose your fighter:

  • '0' means 'no caching'. this is the default level.
  • '1' means 'cache per run'. this caches stemming results during a single call to 'stem'.
  • '2' means 'cache indefinitely'. this caches stemming results until either the process exits or the 'clear_stem_cache' method is called.

🦋 clear_stem_cache

clear_stem_cache;

clears the cache of stemmed words. bye cache. 🫶

📝 NOTES 📝

this code is almost entirely derived from the Porter 2.1 module written by Jim Richardson. iykyk.

👀 SEE ALSO 👀

  • Lingua::Stem

👑 AUTHOR 👑

Jim Richardson, University of Sydney — Jerilyn Franz, FreeRun Technologies.

this code is freely available under the same terms as Perl. 💖

🐛 BUGS 🐛

(the source page has no bug body text — just the heading, bestie. nothing extra to spill.)

📝 TODO 📝

(same situation: the source has the TODO heading and no body text.)

and that's the whole page, bestie! 🫶 this was from perl v5.42.0, dated 2020-09-27, and it's the Lingua::Stem::En(3pm) man page. now go stem something iconic. 💅✨

📦 where this page comes from