ANTLR3_TOKEN_SOURCE_struct(3)

💖 ANTLR3_TOKEN_SOURCE_struct 💖

okay bestie, so this is a page from the ANTLR3C manual — it's literally the whole tea on ANTLR3_TOKEN_SOURCE_struct, no cap. Sit down, grab a strawberry, we're doing struct lore 🍓

---

✨ NAME ✨

ANTLR3_TOKEN_SOURCE_struct — it's the definition of a token source, which has a pointer to a function that returns the next token (using a token factory if it is going to be efficient) and a pointer to an ANTLR3_INPUT_STREAM. that's the vibe, that's the whole thing 💅

---

✨ SYNOPSIS ✨

#include <antlr3tokenstream.h>

🎀 Data Fields 🎀

• ANTLR3_COMMON_TOKEN eofToken — a special pre-allocated token, which signifies End Of Tokens. the finale token. the end credits. 🫶
• pANTLR3_STRING fileName — when the token source is constructed, it is populated with the file name from whence the tokens were produced by the lexer. like a little name tag but for your token source ✨
• pANTLR3_COMMON_TOKEN(* nextToken )(struct ANTLR3_TOKEN_SOURCE_struct *tokenSource) — pointer to a function that returns the next token in the stream. the main character function, fr fr 👑
• ANTLR3_COMMON_TOKEN skipToken — a special pre-allocated token, which is returned by mTokens() if the lexer rule said to just skip the generated token altogether. it's giving "not me, not today" energy 🙈
• pANTLR3_STRING_FACTORY strFactory — whoever is providing tokens, needs to provide a string factory too. package deal, bestie 🎁
• void * super — whatever is supplying the token source interface, needs a pointer to itself so that this pointer can be passed to it when the nextToken function is called. it's the selfie pointer, iconic 📸

---

✨ Detailed Description ✨

Definition of a token source, which has a pointer to a function that returns the next token (using a token factory if it is going to be efficient) and a pointer to an ANTLR3_INPUT_STREAM.

This is slightly different to the Java interface because we have no way to implement multiple interfaces without defining them in the interface structure or casting (void *), which is too convoluted.

lowkey that's the whole drama: Java said "multiple interfaces, slay" and C said "bestie we are not doing that, it's too convoluted" 💀

---

🌸 Field Documentation 🌸

🌈 ANTLR3_COMMON_TOKEN ANTLR3_TOKEN_SOURCE_struct::eofToken

A special pre-allocated token, which signifies End Of Tokens. Because this must be set up with the current input index and so on, we embed the structure and return the address of it. It is marked as factoryMade, so that it is never attempted to be freed.

so like… do NOT try to free it. it is marked factoryMade and it stays. periodt 💅

Referenced by antlr3LexerNew(), nextTokenStr(), setCharStream(), and tokLT().

🌈 pANTLR3_STRING ANTLR3_TOKEN_SOURCE_struct::fileName

When the token source is constructed, it is populated with the file name from whence the tokens were produced by the lexer. This pointer is a copy of the one supplied by the CharStream (and may be NULL) so should not be manipulated other than to copy or print it.

bestie it may be NULL, and it is a copy, so hands off — only copy it or print it, that's it 🙈

Referenced by setCharStream().

🌈 pANTLR3_COMMON_TOKEN(* ANTLR3_TOKEN_SOURCE_struct::nextToken) (struct ANTLR3_TOKEN_SOURCE_struct *tokenSource)

Pointer to a function that returns the next token in the stream.

Referenced by antlr3LexerNew(), and fillBuffer().

🌈 ANTLR3_COMMON_TOKEN ANTLR3_TOKEN_SOURCE_struct::skipToken

A special pre-allocated token, which is returned by mTokens() if the lexer rule said to just skip the generated token altogether. Having this single token stops us wasting memory by have the token factory actually create something that we are going to SKIP(); anyway.

the way this saves you from wasting memory… we love an efficient queen ✨ (yes the original says "by have", we're keeping it, it's part of the lore 🫶)

Referenced by antlr3LexerNew(), and nextTokenStr().

🌈 pANTLR3_STRING_FACTORY ANTLR3_TOKEN_SOURCE_struct::strFactory

Whoever is providing tokens, needs to provide a string factory too.

Referenced by antlr3LexerNew(), setCharStream(), and toStringSS().

🌈 void* ANTLR3_TOKEN_SOURCE_struct::super

Whatever is supplying the token source interface, needs a pointer to itself so that this pointer can be passed to it when the nextToken function is called.

Referenced by antlr3LexerNew(), nextToken(), and nextTokenStr().

---

💜 Author 💜

Generated automatically by Doxygen for ANTLR3C from the source code.

---

kay that's the whole page bestie, you ate that 🫶 this was Version 3.3.1, dated Tue Jul 29 2025, and yes the Doxygen just generated the author section automatically like a little robot fairy 🧚 no notes, mwah 💖