grammar::me_ast(3tcl)

📦
version 2.0 · 2 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

✨ NAME ✨

grammar::me_ast — the many gorgeous little outfits an AST can wear 💅

So like, hi bestie, welcome to the grammar::me_ast manual page from tcllib. This is the doc that spills the tea on all the different representations for abstract syntax trees (AST for short, we're not typing all that out every time 🫶) that get generated by instances of ME virtual machines, and it's written to be independent of which variant you're running. If you have zero clue what a ME virtual machine even is, no worries — please go read grammar::me_intro first, that's the prequel and it ate. 🌸

💖 DESCRIPTION 💖

ASTs — and honestly every representation we're about to specify — split their nodes into two whole categories, and it's giving very binary:

  • Terminal — these refer to the terminal symbols chillin' in the token stream. They are always leaf nodes. Like, a terminal node never has children, periodt. No kids, no chaos, just vibes. 🍓
  • Nonterminal — these represent a nonterminal symbol of the grammar that was used during parsing. They can show up as leaf nodes or as inner nodes of the tree. Main character energy, honestly. 👑

Both node types carry basic range information that tells you which parts of the input the node is covering, by handing you the location of the first and last tokens found within that range. Locations come as non-negative integer offsets from the beginning of the token stream, and the very first token in the stream lives at offset 0 (zero). Not 1. Zero. Remember it. ✨

Oh and btw, the root of an AST can be either a terminal or a nonterminal node. It's flexible like that. 💫

🌈 AST VALUES 🌈

This representation is a Tcl list. The main list is the root node of the tree, and the representations of the children are nested up inside it. It's giving nesting dolls but make it syntax. 🎀

Each node is represented by a single Tcl list holding three or more elements:

  1. The first element is either the empty string, or the name of a nonterminal symbol (and heads up: that name is never the empty string).
  2. The second and third elements are the locations of the first and last tokens.
  3. Any additional elements after the third are the representations of the children — leftmost child first, which means it lands as the fourth element of the list representing the node.

🦋 AST OBJECTS 🦋

In this representation, an AST is represented by a Tcl object command whose API is compatible with the tree objects that the package struct::tree hands out. So basically it has to support at least all of the methods described by that package, and it's allowed to support more if it wants to flex. 💪

Because of that, the rest of these specifications are written using the vocabulary of struct::tree. Iykyk.

Every node of the AST maps directly to a node in the tree object. All the data beyond the child nodes — so, node type and input locations — gets stored in attributes of the node in the tree object. Here they are, bestie:

  • type — the type of the AST node. The recognized values are terminal and nonterminal.
  • range — the locations of the first and last token of the terminal data in the input covered by the node. It's a list containing two locations.
  • detail — this attribute is present only for nonterminal nodes. It holds the name of the nonterminal symbol stored in the node.

🧚 EXTENDED AST OBJECTS 🧚

Extended AST objects are exactly like AST objects, except they come with extra information. Glow-up complete. ✨

  • detail — this attribute is now present at all nodes. Its contents are unchanged for nonterminal nodes. For terminal nodes it contains a list describing all tokens from the input which are covered by the node.

Each element of that list contains the token name, the associated lexeme attribute, line number, and column index — in this order. Keep the order, don't shuffle it. 🙈

  • range_lc — this brand new attribute is defined for all nodes, and it contains the locations from the attribute range translated into line number and column index. Lines are counted from 1, columns are counted from 0. Yes, they start at different numbers. We don't make the rules. 😭

💀 BUGS, IDEAS, FEEDBACK 💀

This document, and the package it describes, will undoubtedly contain bugs and other problems. Like, no cap, it's software. Please report such things in the category grammar_me of the Tcllib Trackers at [http://core.tcl.tk/tcllib/reportlist]. And please also report any ideas for enhancements you may have for either the package and/or the documentation — we wanna hear it. 🫶

When proposing code changes, please provide unified diffs, i.e the output of diff -u.

Also note further that attachments are strongly preferred over inlined patches. You can make an attachment by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar. Slay. 💅

⭐ KEYWORDS ⭐

  • AST
  • abstract syntax tree

🌸 CATEGORY 🌸

Grammars and finite automata

Copyright (c) 2005 Andreas Kupries <andreas_kupries@users.sourceforge.net>

And that's the whole page, bestie — this one's from tcllib, version 0.2, and it lives at grammar::me_ast(3tcl). Go build some trees, you've got this. 💖✨🎀

📦 where this page comes from