Algori...veBayes(3pm)

🎀 NAME 🎀

Algorithm::NaiveBayes — Bayesian prediction of categories ✨

(and yes bestie, this is the 3pm page, i.e. the perl module man page for it — you're in the right place 💅)

---

🦋 SYNOPSIS 🦋

use Algorithm::NaiveBayes;
my $nb = Algorithm::NaiveBayes->new;

$nb->add_instance
  (attributes => {foo => 1, bar => 1, baz => 3},
   label => 'sports');

$nb->add_instance
  (attributes => {foo => 2, blurp => 1},
   label => ['sports', 'finance']);

... repeat for several more instances, then:
$nb->train;

# Find results for unseen instances
my $result = $nb->predict
  (attributes => {bar => 3, blurp => 2});

that's the whole vibe in like ten lines, obsessed 🫶

---

💖 DESCRIPTION 💖

So this module implements the classic "Naive Bayes" machine learning algorithm. It's a super well-studied probabilistic algorithm, and people use it all the time for automatic text categorization. Compared to other algorithms — think kNN, SVM, Decision Trees — it's pretty fast and reasonably competitive in the quality of its results. no cap.

There's a paper by Fabrizio Sebastiani that's a genuinely great intro to text categorization:

<http://faure.iei.pi.cnr.it/~fabrizio/Publications/ACMCS02.pdf>

go read it, it ate 🔥

---

✨ METHODS ✨

🌸 new()

Creates a new Algorithm::NaiveBayes object and hands it right back to you. The following parameters are accepted:

🍓 purge

If you set it to a true value, then the do_purge() method gets invoked during train(). The default is true. Set it to a false value if you'd like to be able to add additional instances after training and then call train() again. iykyk 💅

---

🌈 add_instance( attributes => HASH, label => STRING|ARRAY )

Adds a training instance to the categorizer. The attributes parameter holds a hash reference whose keys are string attributes and whose values are the weights of those attributes. For example, if you're categorizing text documents, the attributes might be the words of the document, and the weights might be the number of times each word occurs in the document. literally so elegant.

The label parameter can contain a single string or an array of strings, and each string represents a label for this instance. The labels can be any arbitrary strings. And if you want to say that a document has no applicable labels, you just pass an empty array reference. main character energy: sometimes you belong to nothing 🫶

---

🧚 train()

Calculates the probabilities that will be necessary for categorization using the predict() method. the glow-up moment ✨

---

👑 predict( attributes => HASH )

Use this method to predict the label of an unknown instance. The attributes should be in the same format as the ones you passed to add_instance(). predict() returns a hash reference whose keys are the names of labels, and whose values are the score for each label. Scores are between 0 and 1, where 0 means the label doesn't seem to apply to this instance, and 1 means it does.

In practice, scores using Naive Bayes tend to be very close to 0 or 1 because of the way normalization is performed. The author says he might try to alleviate this in future versions of the code. (so like, noted, it's a known thing, don't come for him 😭)

---

💜 labels()

Returns a list of all the labels the object knows about (in no particular order), or the number of labels if you call it in a scalar context. context is everything bestie 💅

---

🦋 do_purge()

Purges training instances and their associated information from the NaiveBayes object. This can save memory after training. declutter queen 🧹

---

🎀 purge()

Returns true or false depending on the value of the object's purge property. An optional boolean argument sets the property.

---

⭐ save_state($path)

This object method saves the object to disk for later use. The $path argument indicates the place on disk where the object should be saved:

$nb->save_state($path);

---

🌸 restore_state($path)

This class method reads the file specified by $path and returns the object that was previously stored there using save_state():

$nb = Algorithm::NaiveBayes->restore_state($path);

save her, restore her, she's iconic 💖

---

🔥 THEORY 🔥

Bayes' Theorem is a way of inverting a conditional probability. It states:

                  P(y|x) P(x)
        P(x|y) = -------------
                       P(y)

The notation P(x|y) means "the probability of x given y." See also "/mathforum.org/dr.math/problems/battisfore.03.22.99.html"" in "http: for a simple but complete example of Bayes' Theorem.

In this case, we want to know the probability of a given category given a certain string of words in a document, so we have:

                        P(words | cat) P(cat)
      P(cat | words) = --------------------
                               P(words)

We have applied Bayes' Theorem because P(cat | words) is a difficult quantity to compute directly, but P(words | cat) and P(cat) are accessible (see below).

The greater the expression above, the greater the probability that the given document belongs to the given category. So we want to find the maximum value. We write this as

                                     P(words | cat) P(cat)
      Best category =   ArgMax      -----------------------
                       cat in cats          P(words)

Since P(words) doesn't change over the range of categories, we can get rid of it. That's good, because we didn't want to have to compute these values anyway. So our new formula is:

      Best category =   ArgMax      P(words | cat) P(cat)
                       cat in cats

Finally, we note that if w1, w2, ... wn are the words in the document, then this expression is equivalent to:

      Best category =   ArgMax      P(w1|cat)*P(w2|cat)*...*P(wn|cat)*P(cat)
                       cat in cats

That's the formula used in the document categorization code. The last step is the only non-rigorous one in the derivation, and this is the "naive" part of the Naive Bayes technique. It assumes that the probability of each word appearing in a document is unaffected by the presence or absence of each other word in the document. We assume this even though we know this isn't true: for example, the word "iodized" is far more likely to appear in a document that contains the word "salt" than it is to appear in a document that contains the word "subroutine". Luckily, as it turns out, making this assumption even when it isn't true may have little effect on our results, as the following paper by Pedro Domingos argues: "/www.cs.washington.edu/homes/pedrod/mlj97.ps.gz"" in "http:

💀 not the manpage URL markup being a little cursed — keeping it exactly as-is because facts don't change, only vibes do.

---

🫶 HISTORY 🫶

The first implementation of a Naive Bayes algorithm was in the now-obsolete AI::Categorize module, first released in May 2001. It was replaced with the Naive Bayes implementation in AI::Categorizer (note the extra 'r'), first released in July 2002. Then that implementation got extracted into its own module that could be used outside the framework — and that's what you see here. a whole evolution arc, we love a glow-up 🦋

---

👑 AUTHOR 👑

Ken Williams, ken@mathforum.org

---

💜 COPYRIGHT 💜

Copyright 2003-2004 Ken Williams. All rights reserved.

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

---

🌈 SEE ALSO 🌈

AI::Categorizer(3), perl.

---

and that's the whole page, bestie — you're officially Naive Bayes literate 💅✨

this was perl v5.36.0, dated 2022-11-20, from the Algori...veBayes(3pm) page — mwah, go train something 🎀💖