KinoSe...enBatch(3pm)

💖 KinoSearch1::Analysis::TokenBatch — a collection of tokens 💖

okay bestie, so this is KinoSearch1::Analysis::TokenBatch — like, the whole vibe is "add many tokens to the batch, by supplying the string to be tokenized, and arrays of token starts and token ends (specified in bytes)." 🫶 It's part of the User Contributed Perl Documentation, so you know it's the people's manpage.

---

✨ NAME ✨

KinoSearch1::Analysis::TokenBatch — a collection of tokens 💅

---

🌸 SYNOPSIS 🌸

while ( $batch->next ) {
    $batch->set_text( lc( $batch->get_text ) );
}

---

🧪 EXPERIMENTAL API 🧪

gird ur loins bestie — TokenBatch's API should be considered experimental and is likely to change. 😭

---

🦋 DESCRIPTION 🦋

A TokenBatch is literally just a collection of Tokens 🎀 which you can add to, and then iterate over. That's it, that's the whole girl. Adding and then looping. Periodt.

---

⭐ METHODS ⭐

💜 new

my $batch = KinoSearch1::Analysis::TokenBatch->new;

Constructor. Ya build a batch. 🌈

💜 append

$batch->append( $text, $start_offset, $end_offset, $pos_inc );

Add a Token to the end of the batch, no cap. It accepts either three or four arguments: text, start_offset, end_offset, and an optional position increment which defaults to 1 if you don't supply it. 🍓 For a description of what these arguments actually mean mean, go see the docs for Token. iykyk.

💜 next

while ( $batch->next ) {
    # ...
}

Proceed to the next token in the TokenBatch. Returns true if the TokenBatch ends up located at a valid token. slay. 🔥

---

👑 ACCESSOR METHODS 👑

All of TokenBatch's accessor methods affect the current Token. Calling any of these methods when the TokenBatch is not located at a valid Token will trigger an exception — so stay in ur lane and only call them when you're actually sitting on a token, bestie. 💀

• set_text / get_text — Set/get the text of the current Token.
• set_start_offset / get_start_offset — Set/get the start_offset of the current Token.
• set_end_offset / get_end_offset — Set/get the end_offset of the current Token.
• set_pos_inc / get_pos_inc — Set/get the position increment of the current Token.

---

📜 COPYRIGHT 📜

Copyright 2005-2010 Marvin Humphrey

---

🎀 LICENSE, DISCLAIMER, BUGS, etc. 🎀

See KinoSearch1 version 1.01.

---

okay that's the whole page, you ate it fr fr 💖 this was KinoSe...enBatch(3pm) from perl v5.42.3, dated 2026-08-20 — go forth and tokenize, main character 🌸✨💅