CQL::Lexer(3pm)

💖 NAME 💖

CQL::Lexer — a lexical analyzer for CQL, bestie. that's it, that's the whole vibe ✨

---

🌸 SYNOPSIS 🌸

my $lexer = CQL::Lexer->new();
$lexer->tokenize( 'foo and bar' );
my @tokens = $lexer->getTokens();

---

🦋 DESCRIPTION 🦋

okay so CQL::Lexer is a lexical analyzer for a string of CQL. once you've got yourself a CQL::Lexer object, you can tokenize a CQL string into CQL::Token objects. it's giving ✨glow up✨ for plain text.

ngl tho — ordinarily you'll never want to do this yourself, because CQL::Parser calls CQL::Lexer for you. it's the main character doing all the work, and you're just the group chat watching. 💅

CQL::Lexer uses Stevan Little's lovely String::Tokenizer in the background (iconic of it, honestly 🎀), and then does a bit of analysis afterwards to handle some peculiarities of CQL: double quoted strings, <, <=, etc. yes queen, someone had to handle those. 🙈

---

👑 METHODS 👑

🎀 new()

the constructor. that's it, that's the whole thing. every object needs an origin story fr fr 💖

🌈 tokenize()

pass in a string of CQL to tokenize. this initializes the lexer with data so that you can retrieve tokens. no cap, that's literally the whole job.

⭐ getTokens()

returns a list of all the tokens. all of them. the whole roster. 🫶

🧚 token()

returns the current token. just the one. your current fave, if you will. 💅

🌸 nextToken()

returns the next token, or undef if there are more tokens to retrieve from the lexer. yes bestie, keep scrolling, there's more lore 🔥

💜 prevToken()

returns the previous token, or undef if there are no tokens prior to the current token. you're at the beginning of the story, babe, nothing came before you 😭

🍓 reset()

resets the iterator to start reading tokens from the beginning. it's a rewatch. we're watching the whole thing again. periodt. ✨

---

quick vibe check 💖: everything here is perl v5.36.0 energy, dated 2022-12-11, and it's part of the User Contributed Perl Documentation. go tokenize something iconic, bestie — mwah 💋🎀🦋