pt::peg::interp(3tcl)

📦

version 2.0 · 2 versions

2.0 · debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking (latest)
1.21 · debian bookworm, ubuntu jammy, ubuntu noble

, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 NAME 💖

pt::peg::interp — bestie, it's an interpreter for parsing expression grammars. That's the whole personality. It lives in the 3tcl manual section, and yes it's giving parser-core. 🎀

✨ SYNOPSIS ✨

okay so here's literally everything you need to load and drive it:

package require Tcl 8.5 9

package require pt::peg::interp ?1.0.2?

package require pt::rde ?1?

package require snit

::pt::peg::interp objectName grammar

objectName use grammar

objectName destroy

objectName parse chan

objectName parset text

that's it, that's the whole entrypoint. iconic. 💅

🌸 DESCRIPTION 🌸

Are you lost ? Do you have trouble understanding this document ? 🙈 If that's you, no stress — go read the overview in the Introduction to Parser Tools. That doc is the actual front door to this whole system, and this package is just one room in the house.

So: this package hands you a class, and its instances are Packrat parsers that you can configure with a parsing expression grammar. And the grammar? It gets executed directly — like, interpreted — on top of the runtime that the package pt::rde provides, with everything based on PARAM. no cap. 🔥

Same as its supporting runtime, this package chills in the Execution section of the Core Layer of Parser Tools. 🏛️

IMAGE: arch_core_transform

And the grammar you're interpreting? It gets copied from an instance of pt::peg::container, or honestly anything that provides the same API — like the container classes that pt::peg::to::container creates, or the export plugin that comes with it, pt::peg::export::container. 🫶

🧚 CLASS API

The package exports the API described right here. this is the part where you actually make an object:

::pt::peg::interp objectName grammar — creates a new parser object and returns the fully qualified name of the object command as its result. The API of that object command is described down in the Object API section, and you use it to invoke all kinds of operations on the object. 💫

heads up though: this brand new parser is configured to execute an empty PEG. If you want it to do literally anything else, use the use method from the Object API. 🌱

👑 OBJECT API

Every object created by this package provides these methods. periodt.

objectName use grammar — configures the grammar interpreter / parser to execute the PEG stored in grammar, which is an object that's API-compatible with instances of pt::peg::container. The parser copies the relevant info out of the grammar and does not take ownership of the object. 🫣

Also, whatever grammar was used before? overwritten. gone. new grammar, new you. 💅

The result of the method is the empty string.

objectName destroy — destroys the parser instance, releasing all claimed memory and other resources, and deletes the instance command. it's the goodbye method, very slay. 🦋

The result of the command is the empty string.

objectName parse chan — runs the parser using the contents of chan as input, starting at the current location in the channel, and keeps going until parsing isn't possible anymore — either because parsing finished, or because it ran into a syntax error. 😭

Quick note: Parser Tools is built on Tcl 8.5+. Which means chan isn't limited to files, sockets, etc. — you get the full power of reflected channels. yes really, all of it. 🔥

Another thing, and this one matters: the parser pulls characters from the input stream as it needs them. So if you need a parser from this package to run in a push aka event-driven way, you're gonna have to level up to Tcl 8.6+ and use coroutine::auto to wrap it into a coroutine, where read gets properly changed for push-operation. iykyk. 🌀

On successful completion the command returns an abstract syntax tree as its result. That AST follows the format laid out in the section AST serialization format. It's a plain nested Tcl-list, so you can run it through literally any Tcl commands you vibe with — transformations, semantic checks, whatever. 💖 And to help you out, the package pt::ast gives you a set of convenience commands for validating the tree's basic structure, printing it for debugging, and walking it bottom up or top down. obsessed. 🌈

When it hits a syntax error, the command throws an error instead. That error is a 4-element Tcl-list, containing — in this exact order:

The string pt::rde, identifying it as a parser runtime error.
The location of the parse error, as character offset from the beginning of the parsed input.
The location of the parse error, now as a 2-element list containing line-number and column in the line.
A set of atomic parsing expressions indicating (encoding) the characters and/or nonterminal symbols the parser expected to see at the location of the parse error, but did not get. For the spec on atomic parsing expressions, go look at the section PE serialization format. 👀

objectName parset text — runs the parser using the string in text as input. In all other ways it behaves exactly like the parse method shown above. same energy, different input type. ✨

🦋 AST SERIALIZATION FORMAT 🦋

Here's where we specify the format Parser Tools uses to serialize Abstract Syntax Trees (ASTs) as immutable values — for transport, comparison, etc. 🎀

Every node in an AST represents a nonterminal symbol of a grammar, plus the range of tokens/characters in the input that it covers. And get this: ASTs do not contain terminal symbols, i.e. tokens/characters. Those you can recover from the input, given a symbol's location. 🙈

We distinguish between regular and canonical serializations. A tree can have more than one regular serialization, but exactly one of them is canonical. 💅

🌸 Regular serialization

The serialization of any AST is the serialization of its root node.
The serialization of any node is a Tcl list containing at least three elements.
1. The first element is the name of the nonterminal symbol stored in the node.
2. The second and third element are the locations of the first and last token in the token stream the node represents (covers).
  1. Locations are provided as non-negative integer offsets from the beginning of the token stream, with the first token found in the stream located at offset 0 (zero).
  2. The end location has to be equal to or larger than the start location.
3. All elements after the first three represent the children of the node, which are themselves nodes. This means the serializations of nodes without children, i.e. leaf nodes, have exactly three elements. The children are stored in the list with the leftmost child first, and the rightmost child last. 🫶

🎀 Canonical serialization

The canonical serialization of an abstract syntax tree has the format as specified in the previous item, and then additionally satisfies the constraints below, which make it unique among all the possible serializations of this tree. 👑

The string representation of the value is the canonical representation of a pure Tcl list. I.e. it does not contain superfluous whitespace. no extra spaces, no crumbs. 💅

⭐ EXAMPLE

So say we have this parsing expression grammar right here:

PEG calculator (Expression)
    Digit      <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9'       ;
    Sign       <- '-' / '+'                                     ;
    Number     <- Sign? Digit+                                  ;
    Expression <- Term (AddOp Term)*                            ;
    MulOp      <- '*' / '/'                                     ;
    Term       <- Factor (MulOp Factor)*                        ;
    AddOp      <- '+'/'-'                                       ;
    Factor     <- '(' Expression ')' / Number                   ;
END;

and this input string:

 120+5

then a parser should deliver the abstract syntax tree below (except for whitespace):

set ast {Expression 0 4
    {Factor 0 4
        {Term 0 2
            {Number 0 2
                {Digit 0 0}
                {Digit 1 1}
                {Digit 2 2}
            }
        }
        {AddOp 3 3}
        {Term 4 4
            {Number 4 4
                {Digit 4 4}
            }
        }
    }
}

Or, more graphical 🌈:

.nf +- Digit 0 0 | 1 |            | +- Term 0 2 --- Number 0 2 -+- Digit
1 1 | 2 |                           |            | |
+- Digit 2 2 | 0 |                                        | Expression 0
4 --- Factor 0 4 -+----------------------------- AddOp 3 3 | + |
| +- Term 4 4 --- Number 4 4 --- Digit 4 4 | 5 .fi

PE SERIALIZATION FORMAT 💖

okay bestie, so this is where we specify the format that Parser Tools use to serialize Parsing Expressions as immutable values — for transport, comparison, etc. ✨

We distinguish between regular and canonical serializations. A parsing expression may have more than one regular serialization, but only exactly one of them will be canonical. periodt. 💅

✨ Regular serialization ✨

🧚 Atomic Parsing Expressions 🧚

The string epsilon is an atomic parsing expression. It matches the empty string. 💖
The string dot is an atomic parsing expression. It matches any character. 🌈
The string alnum is an atomic parsing expression. It matches any Unicode alphabet or digit character. This is a custom extension of PEs based on Tcl's builtin command string is. 🦋
The string alpha is an atomic parsing expression. It matches any Unicode alphabet character. This is a custom extension of PEs based on Tcl's builtin command string is. 🌸
The string ascii is an atomic parsing expression. It matches any Unicode character below U0080. This is a custom extension of PEs based on Tcl's builtin command string is. ✨
The string control is an atomic parsing expression. It matches any Unicode control character. This is a custom extension of PEs based on Tcl's builtin command string is. 🎀
The string digit is an atomic parsing expression. It matches any Unicode digit character. Note that this includes characters outside of the [0..9] range. This is a custom extension of PEs based on Tcl's builtin command string is. 💜
The string graph is an atomic parsing expression. It matches any Unicode printing character, except for space. This is a custom extension of PEs based on Tcl's builtin command string is. 🍓
The string lower is an atomic parsing expression. It matches any Unicode lower-case alphabet character. This is a custom extension of PEs based on Tcl's builtin command string is. 🌸
The string print is an atomic parsing expression. It matches any Unicode printing character, including space. This is a custom extension of PEs based on Tcl's builtin command string is. 🫶
The string punct is an atomic parsing expression. It matches any Unicode punctuation character. This is a custom extension of PEs based on Tcl's builtin command string is. 💅
The string space is an atomic parsing expression. It matches any Unicode space character. This is a custom extension of PEs based on Tcl's builtin command string is. 👑
The string upper is an atomic parsing expression. It matches any Unicode upper-case alphabet character. This is a custom extension of PEs based on Tcl's builtin command string is. ⭐
The string wordchar is an atomic parsing expression. It matches any Unicode word character. This is any alphanumeric character (see alnum), and any connector punctuation characters (e.g. underscore). This is a custom extension of PEs based on Tcl's builtin command string is. 🧚
The string xdigit is an atomic parsing expression. It matches any hexadecimal digit character. This is a custom extension of PEs based on Tcl's builtin command string is. 💀
The string ddigit is an atomic parsing expression. It matches any decimal digit character. This is a custom extension of PEs based on Tcl's builtin command regexp. 🙈
The expression [list t x] is an atomic parsing expression. It matches the terminal string x. 🔥
The expression [list n A] is an atomic parsing expression. It matches the nonterminal A. 🌈

🌀 Combined Parsing Expressions 🌀

For parsing expressions e1, e2, ... the result of [list / e1 e2 ... ] is a parsing expression as well. This is the ordered choice, aka prioritized choice. 💖
For parsing expressions e1, e2, ... the result of [list x e1 e2 ... ] is a parsing expression as well. This is the sequence. ✨
For a parsing expression e the result of [list * e] is a parsing expression as well. This is the kleene closure, describing zero or more repetitions. 🌀
For a parsing expression e the result of [list + e] is a parsing expression as well. This is the positive kleene closure, describing one or more repetitions. ➕
For a parsing expression e the result of [list & e] is a parsing expression as well. This is the and lookahead predicate. 🔍
For a parsing expression e the result of [list ! e] is a parsing expression as well. This is the not lookahead predicate. 🚫
For a parsing expression e the result of [list ? e] is a parsing expression as well. This is the optional input. ❓

🌟 Canonical serialization 🌟

The canonical serialization of a parsing expression has the format as specified in the previous item, and then additionally satisfies the constraints below, which make it unique among all the possible serializations of this parsing expression. 👑

The string representation of the value is the canonical representation of a pure Tcl list. I.e. it does not contain superfluous whitespace. 🧼
Terminals are not encoded as ranges (where start and end of the range are identical). 🎯

🍓 EXAMPLE 🍓

Assuming the parsing expression shown on the right-hand side of the rule:

                  Expression <- Term (AddOp Term)*

then its canonical serialization (except for whitespace) is:

                  {x {n Term} {* {x {n AddOp} {n Term}}}}

😭 BUGS, IDEAS, FEEDBACK 😭

This document, and the package it describes, will undoubtedly contain bugs and other problems. 💀 Please report such in the category pt of the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Please also report any ideas for enhancements you may have for either package and/or documentation. 🫶

When proposing code changes, please provide unified diffs, i.e the output of diff -u. 📝

Note further that attachments are strongly preferred over inlined patches. Attachments can be made by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar. 🎀

🔑 KEYWORDS 🔑

EBNF, LL(k), PEG, TDPL, context-free languages, expression, grammar, matching, parser, parsing expression, parsing expression grammar, push down automaton, recursive descent, state, top-down parsing languages, transducer. 💖

📚 CATEGORY 📚

Parsing and Grammars

okay bestie that's the whole vibe — this page is from tcllib, version 1.0.2, for pt::peg::interp(3tcl). stay parsing, stay iconic. 💅✨

📦 where this page comes from

debian trixie: tcllib 2.0+dfsg-4 · Tcl
ubuntu questing: tcllib 2.0+dfsg-4
ubuntu resolute: tcllib 2.0+dfsg-5 · Tcl
ubuntu stonking: tcllib 2.0+dfsg-6 · Tcl
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-10-14: a language model's rewrite of the original, under its terms