Tcl_ParseCommand(3tcl)

📦
version 9.0.4 · 3 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 Tcl_ParseCommand 💖 — a.k.a. the Tcl Library Procedures page, bestie edition ✨

okay so. this is the one that teaches you how to parse Tcl scripts and expressions, and honestly? it's giving compiler-nerd main character energy and I'm obsessed. 🫶

🌸 NAME 🌸

Tcl_ParseCommand, Tcl_ParseExpr, Tcl_ParseBraces, Tcl_ParseQuotedString, Tcl_ParseVarName, Tcl_ParseVar, Tcl_FreeParse, Tcl_EvalTokensStandard — parse Tcl scripts and expressions 💅

✨ SYNOPSIS ✨

#include <tcl.h>

int
Tcl_ParseCommand(interp, start, numBytes, nested, parsePtr)

int
Tcl_ParseExpr(interp, start, numBytes, parsePtr)

int
Tcl_ParseBraces(interp, start, numBytes, parsePtr, append, termPtr)

int
Tcl_ParseQuotedString(interp, start, numBytes, parsePtr, append, termPtr)

int
Tcl_ParseVarName(interp, start, numBytes, parsePtr, append)

const char *
Tcl_ParseVar(interp, start, termPtr)

Tcl_FreeParse(usedParsePtr)

int
Tcl_EvalTokensStandard(interp, tokenPtr, numTokens)

🎀 ARGUMENTS 🎀

  • Tcl_Interp *interp (out) — lowkey the vibes-holder for error reporting. for every procedure except Tcl_FreeParse and Tcl_EvalTokensStandard, it's used only for error reporting; if it's NULL, then no error messages get left behind after errors. for Tcl_EvalTokensStandard tho, it does more — it determines the context for evaluating the script and is also used for error reporting, and it must not be NULL. 🚨
  • const char *start (in) — pointer to the first character in the string you're gonna parse. no cap, that's it. 🫡
  • Tcl_Size numBytes (in) — how many bytes are in the string to parse, not counting any terminating null character. if it's less than 0, then the script is literally all characters following start up to the first null character. slay, auto-detect. ✨
  • int nested (in) — non-zero means the script is part of a command substitution, so an unquoted close bracket should be treated as a command terminator. if it's zero, close brackets have no special meaning at all. 💀
  • int append (in) — non-zero means *parsePtr already has valid tokens, and the new tokens should be appended to the ones already there. zero means *parsePtr is uninitialized and any info inside it is ignored. this argument is normally 0, just so you know. 🫶
  • Tcl_Parse *parsePtr (out) — points to the structure you're filling in with info about the parsed command, expression, variable name, etc. any previous info in this structure is ignored, unless append is non-zero in a call to Tcl_ParseBraces, Tcl_ParseQuotedString, or Tcl_ParseVarName. 🎀
  • const char **termPtr (out) — if it's not NULL, points to a location where Tcl_ParseBraces, Tcl_ParseQuotedString, and Tcl_ParseVar will store a pointer to the character just after the terminating character (the close-brace, the last character of the variable name, or the close-quote, respectively) if the parse was successful. iykyk. 💅
  • Tcl_Parse *usedParsePtr (in) — points to a structure that was filled in by a previous call to Tcl_ParseCommand, Tcl_ParseExpr, Tcl_ParseVarName, etc. recycling era. ♻️

💜 DESCRIPTION 💜

so these procedures parse Tcl commands, or little pieces of Tcl commands like expressions or references to variables. each one takes a pointer to a script (or a chunk of one) and fills in the structure pointed to by parsePtr with a whole collection of tokens describing the info that got parsed. 🧚

normally they return TCL_OK. but bestie, if an error happens, they return TCL_ERROR, leave an error message in interp's result (as long as interp isn't NULL), and leave nothing in parsePtr. no crumbs. 💀

🌟 Tcl_ParseCommand 🌟

Tcl_ParseCommand parses Tcl scripts. give it a pointer to a script and it parses the first command from it. if the command parsed successfully, Tcl_ParseCommand returns TCL_OK and fills in the structure pointed to by parsePtr with info about the structure of the command (more on that below, keep scrolling bestie). if an error happened while parsing, then TCL_ERROR is returned, an error message is left in interp's result, and no info is left at *parsePtr. 🫠

🌟 Tcl_ParseExpr 🌟

Tcl_ParseExpr parses Tcl expressions. give it a pointer to a script containing an expression and it parses that expression. if it parsed successfully, Tcl_ParseExpr returns TCL_OK and fills in the structure pointed to by parsePtr with info about the structure of the expression (again, details below 💅). if an error happened while parsing, then TCL_ERROR is returned, an error message is left in interp's result, and no info is left at *parsePtr. ✨

🌟 Tcl_ParseBraces 🌟

Tcl_ParseBraces parses a string or command argument enclosed in braces — think {hello} or {string \t with \t tabs} — starting from the beginning of its start argument. the first character of start must be {. if the braced string parsed successfully, Tcl_ParseBraces returns TCL_OK, fills in the structure pointed to by parsePtr with info about the structure of the string (yes, details below 🎀), and stores a pointer to the character just after the terminating } in the location given by *termPtr. if an error happens while parsing, then TCL_ERROR is returned, an error message is left in interp's result, and no info is left at *parsePtr or *termPtr. 😭

🌟 Tcl_ParseQuotedString 🌟

Tcl_ParseQuotedString parses a double-quoted string like "sum is [expr {$a+$b}]" from the beginning of the start argument. the first character of start must be ". if the double-quoted string parsed successfully, Tcl_ParseQuotedString returns TCL_OK, fills in the structure pointed to by parsePtr with info about the structure of the string (details below, bestie 💜), and stores a pointer to the character just after the terminating " in the location given by *termPtr. if an error happens while parsing, then TCL_ERROR is returned, an error message is left in interp's result, and no info is left at *parsePtr or *termPtr. 🫠

🌟 Tcl_ParseVarName 🌟

Tcl_ParseVarName parses a Tcl variable reference like $abc or $x([expr {$index + 1}]) from the beginning of its start argument. the first character of start must be $. if a variable name parsed successfully, Tcl_ParseVarName returns TCL_OK and fills in the structure pointed to by parsePtr with info about the structure of the variable name (details below 🎀). if an error happens while parsing, then TCL_ERROR is returned, an error message is left in interp's result (if interp isn't NULL), and no info is left at *parsePtr. 💀

🌟 Tcl_ParseVar 🌟

Tcl_ParseVar parses a Tcl variable reference like $abc or $x([expr {$index + 1}]) from the beginning of its start argument. the first character of start must be $. if the variable name parses successfully, Tcl_ParseVar returns a pointer to the string value of the variable. if an error happens while parsing, then NULL is returned and an error message is left in interp's result. 😤

🧹 memory management moment 🧹

the info left at *parsePtr by Tcl_ParseCommand, Tcl_ParseExpr, Tcl_ParseBraces, Tcl_ParseQuotedString, and Tcl_ParseVarName may include dynamically allocated memory. if these five parsing procedures return TCL_OK, then the caller must invoke Tcl_FreeParse to release the storage at *parsePtr. 🫶

these procedures ignore any existing info in *parsePtr (unless append is non-zero), so if you're making repeated calls to any of them, then Tcl_FreeParse must be invoked once after each call. don't leak, bestie, it's not the vibe. 🚫💧

⚡ Tcl_EvalTokensStandard ⚡

Tcl_EvalTokensStandard evaluates a sequence of parse tokens from a Tcl_Parse structure. the tokens typically consist of all the tokens in a word, or all the tokens that make up the index for a reference to an array variable. Tcl_EvalTokensStandard performs the substitutions the tokens are asking for and concatenates the resulting values. ✨

the return value from Tcl_EvalTokensStandard is a Tcl completion code with one of the values TCL_OK, TCL_ERROR, TCL_RETURN, TCL_BREAK, or TCL_CONTINUE, or possibly some other integer value originating in an extension. 👀 in addition, a result value or error message is left in interp's result, and you can grab it using Tcl_GetObjResult. 🎀

🦋 TCL_PARSE STRUCTURE 🦋

Tcl_ParseCommand, Tcl_ParseExpr, Tcl_ParseBraces, Tcl_ParseQuotedString, and Tcl_ParseVarName return parse info in two data structures: Tcl_Parse and Tcl_Token: 💅

typedef struct {
    const char *commentStart;
    Tcl_Size commentSize;
    const char *commandStart;
    Tcl_Size commandSize;
    Tcl_Size numWords;
    Tcl_Token *tokenPtr;
    Tcl_Size numTokens;
    ...
} Tcl_Parse;

typedef struct {
    int type;
    const char *start;
    Tcl_Size size;
    Tcl_Size numComponents;
} Tcl_Token;

the first five fields of a Tcl_Parse structure are filled in only by Tcl_ParseCommand. those fields are not used by the other parsing procedures. so if you're calling Tcl_ParseExpr, don't be looking at commentStart like 👀 bestie those are not your fields. 🙈

okay so the type field can be exactly ONE of these, and honestly? every single one is a little legend in its own right ✨

TCL_TOKEN_WORD 💖

this token is usually describing one word of a command, but bestie it can also be out here describing a quoted or braced string in an expression, versatile queen. it describes a component of the script that's the result of concatenating a whole sequence of subcomponents together, each one described by its own separate subtoken. the token starts with the first non-blank character of the component (which, plot twist, may be a double-quote or an open brace) and it includes all the characters in the component up to but not including the space, semicolon, close bracket, close quote, or close brace that terminates the component. and the numComponents field counts the total number of sub-tokens that make up the word, including sub-tokens of TCL_TOKEN_VARIABLE and TCL_TOKEN_BS tokens. she contains multitudes, no cap 🫶

TCL_TOKEN_SIMPLE_WORD 🌸

literally the same meaning as TCL_TOKEN_WORD, except the word is guaranteed to consist of a single TCL_TOKEN_TEXT sub-token. and numComponents is always 1. simple, clean, low-maintenance, we love her 🌈

TCL_TOKEN_EXPAND_WORD 🔥

same meaning as TCL_TOKEN_WORD again, BUT the command parser has noted that this word began with the expansion prefix {*}, which means that after substitution, the list value of this word should be expanded to form multiple arguments in command evaluation. iconic behaviour honestly. also this token type can only be created by Tcl_ParseCommand — nobody else gets to make one, periodt 👑

TCL_TOKEN_TEXT 🎀

the token describes a range of literal text that is part of a word. numComponents is always 0. chill, no drama, just text 💜

TCL_TOKEN_BS 🦋

the token describes a backslash sequence, like \n or \0xA3. numComponents is always 0 🍓

TCL_TOKEN_COMMAND ⭐

the token describes a command whose result must be substituted into the word. it includes the square brackets that surround the command. numComponents is always 0 — because the nested command is not parsed. if you want to see its tokens, bestie you have to call Tcl_ParseCommand recursively yourself. do the work, get the tokens 💅

TCL_TOKEN_VARIABLE 💖

the token describes a variable substitution, including the $, the variable name, and the array index (if there is one) up through the close parenthesis that terminates the index. this token is followed by one or more additional tokens that describe the variable name and array index.

  • if numComponents is 1 then the variable is a scalar and the next token is a TCL_TOKEN_TEXT token that gives the variable name.
  • if numComponents is greater than 1 then the variable is an array: the first sub-token is a TCL_TOKEN_TEXT token giving the array name, and the remaining sub-tokens are TCL_TOKEN_TEXT, TCL_TOKEN_BS, TCL_TOKEN_COMMAND, and TCL_TOKEN_VARIABLE tokens that must be concatenated to produce the array index.

and numComponents includes nested sub-tokens that are part of TCL_TOKEN_VARIABLE tokens in the array index. layers on layers on layers, we stan a recursive queen 👑

TCL_TOKEN_SUB_EXPR 🧚

the token describes one subexpression of an expression (or an entire expression, main character energy). a subexpression may consist of a value, like an integer literal, a variable substitution, or a parenthesized subexpression; it may also consist of an operator and its operands. the token starts with the first non-blank character of the subexpression, up to but not including the space, brace, close-paren, or bracket that terminates the subexpression.

this token is followed by one or more additional tokens that describe the subexpression. here's the tea:

  • if the first sub-token after the TCL_TOKEN_SUB_EXPR token is a TCL_TOKEN_OPERATOR token, then the subexpression consists of an operator and its token operands.
  • if the operator has no operands, the subexpression consists of just the TCL_TOKEN_OPERATOR token.
  • each operand is described by a TCL_TOKEN_SUB_EXPR token.
  • otherwise, the subexpression is a value described by one of the token types TCL_TOKEN_WORD, TCL_TOKEN_TEXT, TCL_TOKEN_BS, TCL_TOKEN_COMMAND, TCL_TOKEN_VARIABLE, and TCL_TOKEN_SUB_EXPR.

the numComponents field counts the total number of sub-tokens that make up the subexpression — and yes that includes the sub-tokens for any nested TCL_TOKEN_SUB_EXPR tokens 💜

TCL_TOKEN_OPERATOR 🌟

the token describes one operator of an expression, like && or hypot. a TCL_TOKEN_OPERATOR token is always preceded by a TCL_TOKEN_SUB_EXPR token that describes the operator and its operands, and you can use that TCL_TOKEN_SUB_EXPR token's numComponents field to figure out how many operands there are. iykyk 💅

  • a binary operator like * is followed by two TCL_TOKEN_SUB_EXPR tokens that describe its operands.
  • a unary operator like - is followed by a single TCL_TOKEN_SUB_EXPR token for its operand.
  • if the operator is a math function like log10, the TCL_TOKEN_OPERATOR token gives its name and the following TCL_TOKEN_SUB_EXPR tokens describe its operands. if there are no operands (as with rand), then no TCL_TOKEN_SUB_EXPR tokens follow.
  • there is exactly one trinary operator, ?, and it shows up in if-then-else subexpressions like x?y:z. in that case, the ? TCL_TOKEN_OPERATOR token is followed by three TCL_TOKEN_SUB_EXPR tokens for the operands x, y, and z.

and the numComponents field for a TCL_TOKEN_OPERATOR token is always 0. zero. nothing. she gives nothing and takes everything 🙈

✨ what you get back after each parse ✨

after Tcl_ParseCommand returns, the first token pointed to by the tokenPtr field of the Tcl_Parse structure always has type TCL_TOKEN_WORD or TCL_TOKEN_SIMPLE_WORD or TCL_TOKEN_EXPAND_WORD. it's followed by the sub-tokens that must be concatenated to produce the value of that word. then the next token is the TCL_TOKEN_WORD or TCL_TOKEN_SIMPLE_WORD or TCL_TOKEN_EXPAND_WORD token for the second word, followed by sub-tokens for that word, and so on until all numWords have been accounted for. it's a whole parade of words, one after another 🎀

after Tcl_ParseExpr returns, the first token pointed to by the tokenPtr field of the Tcl_Parse structure always has type TCL_TOKEN_SUB_EXPR. it's followed by the sub-tokens that must be evaluated to produce the value of the expression. and heads up bestie: only the token information in the Tcl_Parse structure is modified — the commentStart, commentSize, commandStart, and commandSize fields are not modified by Tcl_ParseExpr.

after Tcl_ParseBraces returns, the array of tokens pointed to by the tokenPtr field of the Tcl_Parse structure will contain a single TCL_TOKEN_TEXT token if the braced string does not contain any backslash-newlines. if the string does contain backslash-newlines, the array of tokens will contain one or more TCL_TOKEN_TEXT or TCL_TOKEN_BS sub-tokens that must be concatenated to produce the value of the string. and if the braced string was just {} (aka the string was empty), the single TCL_TOKEN_TEXT token will have a size field containing zero — this ensures that at least one token appears to describe the braced string. again, only the token information in the Tcl_Parse structure is modified: the commentStart, commentSize, commandStart, and commandSize fields are not modified by Tcl_ParseBraces.

after Tcl_ParseQuotedString returns, the array of tokens pointed to by the tokenPtr field of the Tcl_Parse structure depends on the contents of the quoted string. it will consist of one or more TCL_TOKEN_TEXT, TCL_TOKEN_BS, TCL_TOKEN_COMMAND, and TCL_TOKEN_VARIABLE sub-tokens. the array always contains at least one token; for example, if the argument start is empty, the array returned consists of a single TCL_TOKEN_TEXT token with a zero size field. and once more for the people in the back: only the token information in the Tcl_Parse structure is modified — the commentStart, commentSize, commandStart, and commandSize fields are not modified.

after Tcl_ParseVarName returns, the first token pointed to by the tokenPtr field of the Tcl_Parse structure always has type TCL_TOKEN_VARIABLE. it's followed by the sub-tokens that make up the variable name, as described above. the total length of the variable name is contained in the size field of the first token. and just like in Tcl_ParseExpr, only the token information in the Tcl_Parse structure is modified by Tcl_ParseVarName: the commentStart, commentSize, commandStart, and commandSize fields are not modified 🦋

all of the character pointers in the Tcl_Parse and Tcl_Token structures refer to characters in the start argument passed to Tcl_ParseCommand, Tcl_ParseExpr, Tcl_ParseBraces, Tcl_ParseQuotedString, and Tcl_ParseVarName. don't be out here thinking they point somewhere else, they don't 💀

and okay, last lil warning: there are additional fields in the Tcl_Parse structure after the numTokens field, but those are for the private use of Tcl_ParseCommand, Tcl_ParseExpr, Tcl_ParseBraces, Tcl_ParseQuotedString, and Tcl_ParseVarName. they should not be referenced by code outside of these procedures. those fields are not for you, bestie, stay out of their business 🙈

✨ KEYWORDS ✨

backslash substitution, braces, command, expression, parse, token, variable substitution

and that's the whole page, we made it 🎉 this one's from Tcl 8.3, and it's the Tcl_ParseCommand(3tcl) man page — go forth and parse, you gorgeous thing, you're literally the main character of the tokenizer 🌈💖👑

📦 where this page comes from