ANTLR3_RECOGNI...ED_STATE_struct(3)

📦 version 3.4, in fedora 43

💖 ANTLR3_RECOGNI...ED_STATE_struct — part 1 of 2 ✨

okay bestie, pull up a chair, we're doing ANTLR3_RECOGNI...ED_STATE_struct(3) from the ANTLR3C manual. yes the name is absurdly long, no I did not choose it, and yes it is lowkey iconic. this is the big one, the struct that holds everything about where a recognizer is at right now. we're doing part 1 today, part 2 is coming, do not panic. 🦋

🌸 NAME 🌸

ANTLR3_RECOGNIZER_SHARED_STATE_struct — literally all the data elements you need to track the current state of any recognizer (lexer, parser, tree parser). all of them. every last one. she's a keeper. 💅

🎀 SYNOPSIS 🎀

#include <antlr3recognizersharedstate.h>

✨ Data Fields ✨

okay so this is the whole squad, one by one, no skipping, no "oh that one's boring", we love all of them equally:

  • ANTLR3_INT32 backtracking — if it's 0, no backtracking is going on. that's it. that's the field. when she's not 0, something's up. 👀
  • ANTLR3_UINT32 channel — the channel number for the current token. like a TV channel but for tokens. slay.
  • void * custom — (no description on this one bestie, she's mysterious, she's yours to fill in 🙈)
  • ANTLR3_BOOLEAN error — if it gets set to ANTLR3_TRUE then the recognizer is in an exception condition. and yes, the generated code for your grammar's rules actually tests this. so it matters. fr fr.
  • ANTLR3_UINT32 errorCount — when the recognizer terminates, the error handling functions will have bumped this value if any error occurred (that was displayed). so it's the receipt for your mistakes. 😭
  • ANTLR3_BOOLEAN errorRecovery — true when we see an error and before we've successfully matched a token. so like, mid-recovery, mid-arc, main character energy in the healing era.
  • pANTLR3_EXCEPTION exception — points to the first in a possible chain of exceptions the recognizer has discovered. a chain!! she's collecting them. 💀
  • ANTLR3_BOOLEAN failed — in lieu of a return value, this says a rule or token failed to match. not a vibe, but a necessary one.
  • pANTLR3_STACK following — tracks the set of token types that can follow any rule invocation. the "what comes next" tracker. very planner-girl of her. 🎀
  • ANTLR3_MARKER lastErrorIndex — the index into the input stream where the last error occurred. your error's home address. 📍
  • pANTLR3_VECTOR rStreams — a stack of token/tree rewrite streams, available for use by a parser or tree parser that's using rewrites to generate an AST. rewrite girlies, this one's for you. ✨
  • pANTLR3_INT_TRIE ruleMemo — an ANTLR3_VECTOR of ANTLR3_LIST for rule memoizing. memoizing!! she remembers so you don't have to. 🧠
  • ANTLR3_UINT32 sizeHint — tracks around a hint from the creator of the recognizer as to how big this thing is going to get, as the actress said to the bishop. (the manual said that, not me, I'm just the messenger 💀)
  • pANTLR3_STACK streams — input stream stack. it lets the C programmer switch input streams easily, and lets the standard nextToken() implementation deal with it, because apparently that's a common requirement. and honestly? fair. 🌈
  • pANTLR3_STRING text — text for the current token. the token's little outfit. 💅
  • pANTLR3_COMMON_TOKEN token — the goal of all lexer rules/methods is to create a token object. that's the whole point. the token is the point. 👑
  • pANTLR3_UINT8 * tokenNames — pointer to an array of token names that are generally useful in error reporting. so when something breaks you get a name and not just a number. obsessed.
  • ANTLR3_MARKER tokenStartCharIndex — what character index in the stream did the current token start at? just a little "where were you when it began" moment. 🫶
  • ANTLR3_INT32 tokenStartCharPositionInLine — the character position of the first character of the current token, within the line specified by tokenStartLine. coordinates, basically. ✨
  • ANTLR3_INT32 tokenStartLine — the input line (where it makes sense) on which the first character of the current token resides. "where it makes sense" bestie, the manual is being careful, respect it.
  • pANTLR3_TOKEN_FACTORY tokFactory — since the goal of all lexer rules is to create a token, a lexer needs to build a token factory to create them. she's not just making tokens, she's making the machine that makes the tokens. 🏭💖
  • pANTLR3_TOKEN_SOURCE tokSource — a lexer is a source of tokens, produced by all the generated (or hand crafted, if you like) matching rules. yes, hand crafted. artisan tokens. no cap.
  • ANTLR3_UINT32 type — the token type for the current token. simple, clean, essential. ⭐
  • ANTLR3_UINT32 user1 — user controlled variables that will be installed in a newly created token. this is your field, bestie, go wild.
  • ANTLR3_UINT32 user2 — (no description, she's a mystery, she's a sequel 🙈)
  • ANTLR3_UINT32 user3 — (also no description. three of them. a trilogy. 💜)
  • void * userp — user programmable pointer. you can use it, for instance, as a place to store some tracking structure specific to the grammar that wouldn't normally be available to the error handling functions. it's your little secret pocket. 🤫

🌟 Detailed Description 🌟

All the data elements required to track the current state of any recognizer (lexer, parser, tree parser).

May be share between multiple recognizers such that grammar inheritance is easily supported.

okay so that second line is doing a LOT of work — one shared state, many recognizers, grammar inheritance just works. that's the whole flex. ✨

okay that's the data fields, the whole roster, everyone accounted for. part 2 is still coming bestie, don't go anywhere 🌸💅

✨ Field Documentation ✨

okay bestie, this is the main event 💖 every single field in ANTLR3_RECOGNIZER_SHARED_STATE_struct, one by one, with the full receipts on who touches what. no skipping, no skimming, we're doing all of them 🫶

🌸 backtracking

  • Type: ANTLR3_INT32
  • If it's 0, no backtracking is going on — totally safe to exec actions and whatever else. If it's >0, that number is the level of backtracking.
  • Referenced by antlr3BaseRecognizerNew(), match(), matchc(), matchRange(), matchs(), noViableAlt(), reset(), and synpred().

🌸 channel

  • Type: ANTLR3_UINT32
  • The channel number for the current token.
  • Referenced by emit(), nextTokenStr(), and reset().

🌸 custom

  • Type: void*
  • Referenced by emit(), and nextTokenStr().

🌸 error

  • Type: ANTLR3_BOOLEAN
  • If you set it to ANTLR3_TRUE, then the recognizer has an exception condition going on — and yeah, the generated code for your grammar's rules literally tests this.
  • Referenced by antlr3dfapredict(), antlr3RecognitionExceptionNew(), nextTokenStr(), recover(), recoverFromMismatchedElement(), recoverFromMismatchedSet(), and recoverFromMismatchedToken().

🌸 errorCount

  • Type: ANTLR3_UINT32
  • When the recognizer terminates, the error handling functions will have bumped this value if any error happened (and was displayed). So you can use it from your grammar code without resorting to static globals, king behaviour 👑
  • Referenced by antlr3BaseRecognizerNew(), getNumberOfSyntaxErrors(), reportError(), reportError(), and reset().

🌸 errorRecovery

  • Type: ANTLR3_BOOLEAN
  • This is true when we've seen an error and before we've successfully matched a token. It's what stops you from getting more than one error message per error. we love a one-and-done 💅
  • Referenced by antlr3BaseRecognizerNew(), match(), matchAny(), reportError(), and reset().

🌸 exception

  • Type: pANTLR3_EXCEPTION
  • Points to the first in a possible chain of exceptions that the recognizer has discovered. it's giving lore.
  • Referenced by antlr3MTExceptionNew(), antlr3MTNExceptionNew(), antlr3RecognitionExceptionNew(), displayRecognitionError(), displayRecognitionError(), freeBR(), mismatch(), mismatchIsUnwantedToken(), noViableAlt(), recoverFromMismatchedSet(), recoverFromMismatchedToken(), reportError(), and reset().

🌸 failed

  • Type: ANTLR3_BOOLEAN
  • Instead of a return value, this is how it tells you that a rule or token failed to match. It gets reset to false once a valid token matches.
  • Referenced by alreadyParsedRule(), antlr3BaseRecognizerNew(), match(), matchAny(), matchc(), matchRange(), matchs(), memoize(), nextTokenStr(), noViableAlt(), recover(), recoverFromMismatchedElement(), recoverFromMismatchedSet(), reset(), and synpred().

🌸 following

  • Type: pANTLR3_STACK
  • Tracks the set of token types that can follow any rule invocation. It's a stack structure, existing to support: List<BitSet>.
  • Referenced by antlr3BaseRecognizerNew(), combineFollows(), freeParser(), freeParser(), and reset().

🌸 lastErrorIndex

  • Type: ANTLR3_MARKER
  • The index into the input stream where the last error happened. Why does this exist? To prevent infinite loops where an error is found but no token gets consumed during recovery… and then another error is found… ad nauseam 😭 It's a failsafe, guaranteeing that at least one token or tree node gets consumed for two errors.
  • Referenced by antlr3BaseRecognizerNew(), recover(), and reset().

🌸 rStreams

  • Type: pANTLR3_VECTOR
  • A stack of token/tree rewrite streams available for a parser or tree parser that's using rewrites to generate an AST. It saves every rule in the recognizer from having to allocate and deallocate rewrite streams on entry and exit. As the parser recurses through the rules, it'll reach a steady state of the maximum number of allocated streams — and instead of deallocating them at rule exit, it drops them on this stack for reuse. Then everything gets finally freed when this stack is freed. eco-friendly icon behaviour ♻️
  • Referenced by antlr3BaseRecognizerNew(), antlr3RewriteRuleElementStreamNewAE(), freeBR(), freeNodeRS(), and freeRS().

🌸 ruleMemo

  • Type: pANTLR3_INT_TRIE
  • An ANTLR3_VECTOR of ANTLR3_LIST for rule memoizing. It tracks the stop token index for each rule. ruleMemo[ruleIndex] is the memoization table for ruleIndex. For key ruleStartIndex, you get back the stop token for the associated rule, or MEMO_RULE_FAILED.
  • Heads up: this is only used if rule memoization is on.
  • Referenced by antlr3BaseRecognizerNew(), freeBR(), getRuleMemoization(), memoize(), and reset().

🌸 sizeHint

  • Type: ANTLR3_UINT32
  • Tracks around a hint from whoever created the recognizer as to how big this thing is going to get, as the actress said to the bishop 😳 This lets us tune hash tables accordingly. Might not be the best place for this in the end, but we will see.
  • Referenced by antlr3BaseRecognizerNew().

🌸 streams

  • Type: pANTLR3_STACK
  • Input stream stack — it lets the C programmer switch input streams easily, and lets the standard nextToken() implementation deal with it, since that's a common requirement.
  • Referenced by freeLexer(), nextToken(), popCharStream(), and pushCharStream().

🌸 text

  • Type: pANTLR3_STRING
  • The text for the current token. You can override it by setting this variable directly, or by using the SETTEXT() macro (preferred) in your lexer rules. do the preferred one bestie 🎀
  • Referenced by emit(), getText(), nextTokenStr(), reset(), and setCharStream().

🌸 token

  • Type: pANTLR3_COMMON_TOKEN
  • The goal of all lexer rules/methods is to create a token object, and this is where it lives. It's an instance variable because multiple rules may collaborate to create a single token. For example:
NUM : INT | FLOAT ;
  • In that case, you want the INT or FLOAT rule to set token and not have it reset to a NUM token in rule NUM.
  • Referenced by emit(), emitNew(), nextTokenStr(), reset(), and setCharStream().

🌸 tokenNames

  • Type: pANTLR3_UINT8*
  • Pointer to an array of token names that are generally useful in error reporting. The generated parsers install this pointer. The table it points to is statically allocated as 8-bit ascii at parser compile time — so grammar token names are restricted in character sets, which does not seem too terrible.
  • Referenced by antlr3BaseRecognizerNew(), getMissingSymbol(), getMissingSymbol(), reportError(), and reportError().

🌸 tokenStartCharIndex

  • Type: ANTLR3_MARKER
  • What character index in the stream did the current token start at? Needed, for example, to get the text for the current token. Set at the start of nextToken.
  • Referenced by displayRecognitionError(), emit(), getText(), nextTokenStr(), reset(), and setCharStream().

🌸 tokenStartCharPositionInLine

  • Type: ANTLR3_INT32
  • The character position of the first character of the current token within the line specified by tokenStartLine.
  • Referenced by displayRecognitionError(), emit(), nextTokenStr(), and reset().

🌸 tokenStartLine

  • Type: ANTLR3_INT32
  • The input line (where it makes sense) on which the first character of the current token resides.
  • Referenced by displayRecognitionError(), emit(), nextTokenStr(), and reset().

🌸 tokFactory

  • Type: pANTLR3_TOKEN_FACTORY
  • Since the goal of all lexer rules is creating a token, a lexer needs to build a token factory to create them. supply chain, but make it parsing 🏭
  • Referenced by antlr3BaseRecognizerNew(), antlr3LexerNew(), emit(), freeBR(), freeLexer(), getMissingSymbol(), reset(), and setCharStream().

🌸 tokSource

  • Type: pANTLR3_TOKEN_SOURCE
  • A lexer is a source of tokens, produced by all the generated (or hand crafted, if you like) matching rules. As such, it needs to provide a token source interface implementation.
  • Referenced by antlr3BaseRecognizerNew(), antlr3LexerNew(), freeLexer(), and setCharStream().

🌸 type

  • Type: ANTLR3_UINT32
  • The token type for the current token.
  • Referenced by emit(), and reset().

🌸 user1

  • Type: ANTLR3_UINT32
  • User controlled variables that will be installed in a newly created token.
  • Referenced by emit(), and nextTokenStr().

🌸 user2

  • Type: ANTLR3_UINT32
  • Referenced by emit(), and nextTokenStr().

🌸 user3

  • Type: ANTLR3_UINT32
  • Referenced by emit(), and nextTokenStr().

🌸 userp

  • Type: void*
  • A user programmable pointer that can be used for instance as a place to store some tracking structure specific to the grammar — the kind of thing that would not normally be available to the error handling functions. your lil pocket dimension, go off 🦋

👑 Author

Generated automatically by Doxygen for ANTLR3C from the source code. yes, the robot wrote the docs, the humans wrote the parser, we love a collab 💜

okay that's the whole struct, every field accounted for, no cap ✨ this was Version 3.3.1 and it's dated Tue Jul 29 2025 — go forth and dereference responsibly, bestie 🌈💖

📦 where this page comes from