AI::Ca...ocument(3pm)

💖 AI::Categorizer::Document — bestie's guide to the Document class 💖

ok so this is the AI::Categorizer::Document page from the User Contributed Perl Documentation section of your perl man pages. she's a whole vibe and she embodies a document. yes literally. that's the job. 🎀

---

✨ NAME ✨

AI::Categorizer::Document — Embodies a document. that's it. that's the whole personality. she's a document but make it an object. 💅

---

🦋 SYNOPSIS 🦋

here's how you actually use her, in all her forms:

use AI::Categorizer::Document;

# Simplest way to create a document:
my $d = new AI::Categorizer::Document(name => $string,
                                      content => $string);

# Other parameters are accepted:
my $d = new AI::Categorizer::Document(name => $string,
                                      categories => \@category_objects,
                                      content => { subject => $string,
                                                   body => $string2, ... },
                                      content_weights => { subject => 3,
                                                           body => 1, ... },
                                      stopwords => \%skip_these_words,
                                      stemming => $string,
                                      front_bias => $float,
                                      use_features => $feature_vector,
                                     );

# Specify explicit feature vector:
my $d = new AI::Categorizer::Document(name => $string);
$d->features( $feature_vector );

# Now pass the document to a categorization algorithm:
my $learner = AI::Categorizer::Learner::NaiveBayes->restore_state($path);
my $hypothesis = $learner->categorize($document);

no cap, that first one is the quick-and-easy route and the second one is the "i have opinions about my data" route. both valid. both slay. ✨

---

🌸 DESCRIPTION 🌸

so the Document class is the one that embodies the data in a single document, and she comes with methods for turning that data into a FeatureVector. it's giving "data glow-up". ✨

most of the time documents are just plain text, but subclasses of the Document class can handle literally any kind of data you throw at them. flexible queen. 👑

---

💅 METHODS 💅

new(%parameters)

creates a new Document object. 🎀

Document objects get used everywhere, bestie:

• during training (for the training documents)
• during testing (for the test documents)
• when categorizing new unseen documents in an application (for the unseen documents)

but lowkey, you'll usually only call new() in that last case, because the KnowledgeSet or Collection classes will make Document objects for you in the first two cases. so like, don't do extra work. let them cook. 😌

the new() method accepts these parameters:

• name — a string that identifies this document. Required. not optional, bestie, she needs a name. 💖
• content — the raw content of this document. can be either a string, or a hash reference, which is how you do structured document types. ✨
• content_weights — a hash reference saying what weights should be assigned to features in different sections of a structured document when the feature vector gets created. the weight is a multiplier of the feature vector values. for example: if a "subject" section has a weight of 3 and a "body" section has a weight of 1, and word counts are the feature vector values, then it's as if every word appearing in the "subject" appeared 3 times. iconic math honestly. 🧮

if no weights are specified, all weights are set to 1.

• front_bias — lets you smoothly bias the weights of words in a document based on their position. the value should be a number between -1 and 1. 👑
  • positive numbers → words toward the beginning of the document get higher weight than words toward the end
  • negative numbers → the opposite
  • a bias of 0 → no biasing at all
• categories — a reference to an array of Category objects that this document belongs to. optional. 🎀
• stopwords — a list/hash of features (words) that should be ignored when parsing document content. a hash reference is preferred, with the features as the keys. if you pass an array reference containing the features, it gets converted to a hash reference internally. we do the conversion for you, no worries. 💜
• use_features — a Feature Vector specifying the only features that should be considered when parsing this document. this is the alternative to using "stopwords". 🌈
• stemming — says which linguistic procedure should be used to convert tokens in the document into features. possible values:
  • "none" — tokens are used without change
  • "porter" — the Porter stemming algorithm gets applied to each token. heads up: this one requires the "Lingua::Stem" module from CPAN. 📦
• stopword_behavior — okay so there are a few ways you might want the stopword list (that's the "stopwords" parameter) to interact with the stemming algorithm (that's the "stemming" parameter), and this parameter controls all of that. it can take these values:
  • no_stem — match stopwords against non-stemmed document words.
  • stem — stem the stopwords according to the 'stemming' parameter, then match them against stemmed document words.
  • pre_stemmed — stopwords are already stemmed, so match them against stemmed document words.

the default value is "stem", which seems to produce the best results in most cases the author has tried. (they're also not aware of any studies comparing the "no_stem" behavior to the "stem" behavior in the general case. honest king. 🙈)

this parameter has no effect if there are no stopwords being used, or if stemming is not being used. in that latter case, the list of stopwords will always be matched as-is against the document words.

⚠️ important, bestie: if the "stem" option is used, the data structure you passed as the "stopwords" parameter will be modified in-place to contain the stemmed versions of the stopwords you supplied. she will edit your hash. fr fr. 😭

read( path => $path )

an alternative constructor method. reads a file on disk and returns a document with that file's contents. 📖✨

parse( content => $content )

this one's listed right here next to the others. 🎀

name()

returns this document's "name" property, exactly as it was specified when the document was created. 💖

features()

returns the Feature Vector associated with this document. 🦋

categories()

in a list context, returns a list of Category objects that this document belongs to.

in a scalar context, returns the number of such categories. she's versatile like that. 💅

create_feature_vector()

creates this document's Feature Vector by parsing its content. you won't call this method directly — it's called by new(). she's a backstage queen, main character energy but offstage. 👑

---

🌟 AUTHOR 🌟

Ken Williams <ken@mathforum.org> 💜

---

📜 COPYRIGHT 📜

this distribution is free software; you can redistribute it and/or modify it under the same terms as Perl itself. these terms apply to every file in the distribution — if you have questions, please contact the author. 🫶

---

and that's the whole page, bestie!! this was perl v5.42.0 flavored, dated 2025-07-25, and it's the AI::Ca...ocument(3pm) manual page. go forth and categorize. slay. 💅✨🎀🦋💖