grammar::me::cpu(n)

🌸 grammar::me::cpu 🌸

okay bestie, so this is the grammar::me::cpu(n) page — it lives in the Grammar operations and usage corner of the Tcllib manual, and it's all about the ME virtual machine. Virtual machine implementation II for parsing token streams, to be precise. 💅

---

✨ NAME ✨

grammar::me::cpu — Virtual machine implementation II for parsing token streams 🧚

---

🎀 SYNOPSIS 🎀

package require Tcl 8.5 9

package require grammar::me::cpu ?0.3?

::grammar::me::cpu meName matchcode

meName option ?arg arg ...?

meName lc location

meName tok ?from ?to??

meName pc state

meName iseof state

meName at state

meName cc state

meName sv

meName ok

meName error

meName lstk state

meName astk state

meName mstk state

meName estk state

meName rstk state

meName nc state

meName ast

meName halted

meName code

meName eof

meName put tok lex line col

meName putstring string lvar cvar

meName run ?n?

meName pull nextcmd

meName reset

meName destroy

---

💖 DESCRIPTION 💖

So this package hands you an implementation of the ME virtual machine. If you're sitting there like "bestie what even is a ME virtual machine 😭", that's your sign to go read grammar::me_intro first. Fr fr, it's the prerequisite.

This implementation gives you an object-based API, and the machines aren't truly tied to Tcl — like, a C implementation of the exact same API is totally possible. Main character energy across languages. 👑

Under the hood, the package is actually using the value-based machine manipulation commands from grammar::me::cpu::core to do all the work. No cap.

---

🦋 API 🦋

🌟 CLASS API 🌟

The package gives you exactly one command for constructing ME virtual machines. That's it. One. Iconic minimalism.

• ::grammar::me::cpu meName matchcode — makes a brand-new ME machine object, with an associated global Tcl command whose name is meName. You use that command to invoke all the various operations on the machine. Its general form is:

meName option ?arg arg ...?

where option and the args decide exactly what the command does. 💫

The argument matchcode holds the match instructions the machine has to execute while it's parsing the input stream. Wanna know the structure of that value? Go read the section MATCH CODE REPRESENTATION in the docs for the package grammar::me::cpu::core. That's where the tea is.

Oh, and about the tokmap argument — the implementation provided by the package grammar::me::tcl takes that thing and hides it inside the match instructions, so you don't need it. It's giving "don't worry about it, it's handled." 🫶

💫 OBJECT API 💫

Every ME virtual machine object created by the class command from CLASS API supports all the methods listed below. Every. Single. One.

These machines do both push-style and pull-style operation, and honestly? Versatile. ✨ Push-style means tokens get pushed into the machine state as they arrive, which triggers further execution until those tokens are consumed. Translation: the machine can be suspended and resumed whenever you want, an arbitrary number of times — so you get quasi-parallel operation of several machines at once, and you can run it as part of the event loop. Slay.

• meName lc location — converts a token's location, given as an offset into the input stream, into the associated line number and column index. You get back a 2-element list with those two values, in exactly that order (line first, then column). This is the glow-up that lets higher levels turn the location info from error status and generated ASTs into something humans can actually read. 📖

Heads up bestie: this command cannot convert locations the machine hasn't reached yet. Like, if the machine has read 7 tokens, you can convert offsets 0 to 6 and nothing beyond. Also, you can't convert offsets pointing to locations before the beginning of the stream. Boundaries, okay? 💅

• meName tok ?from ?to?? — returns a Tcl list containing the part of the input stream between the locations from and to, both inclusive. No to? It defaults to the value of from. No from either? You get the whole input stream, the entire thing, no crumbs. Each element of the returned list is itself a list of four elements: the token, its associated lexeme, the line number, and the column index, in that order. Same location restrictions as the lc method above — the rules don't change just because you asked nicely.
• meName pc state — takes the state value of a ME virtual machine and gives you back the current value of the stored program counter.
• meName iseof state — takes the state value of a ME virtual machine and gives you back the current value of the stored eof flag.
• meName at state — takes the state value of a ME virtual machine and gives you back the current location in the input stream.
• meName cc state — takes the state value of a ME virtual machine and gives you back the current token.
• meName sv — returns the current semantic value SV stored in the machine. That's an abstract syntax tree, exactly as specified in grammar::me_ast, section AST VALUES. 🌳
• meName ok — returns the current match status OK.
• meName error — returns the current error status ER. (We don't gatekeep errors here. 😭)
• meName lstk state — takes the state value of a ME virtual machine and returns the location stack.
• meName astk state — takes the state value of a ME virtual machine and returns the AST stack.
• meName mstk state — takes the state value of a ME virtual machine and returns the AST marker stack.
• meName estk state — takes the state value of a ME virtual machine and returns the error stack.
• meName rstk state — takes the state value of a ME virtual machine and returns the subroutine return stack.
• meName nc state — takes the state value of a ME virtual machine and returns the nonterminal match cache as a dictionary.
• meName ast — returns the current top entry of the AST stack AS. Also an abstract syntax tree, as specified in grammar::me_ast, section AST VALUES.
• meName halted — returns a boolean telling you whether the engine has halted or not. Halt means no further matching is possible and everything you can grab via the other methods is final. Attempts to run the engine get straight-up ignored until a reset happens. It's in its locked-in era. 🔒
• meName code — returns the code information used to construct the object. In other words: the match program the machine is executing.
• meName eof — adds an end of file marker to the end of the input stream. This signals to the machine that the current contents of the input queue are the final parts of the input and nothing is coming after. Attempts to put more characters into the queue will fail. Periodt. 🛑
• meName put tok lex line col — adds the token tok to the end of the input stream, with associated lexeme data lex and line/column information.
• meName putstring string lvar cvar — adds each individual character in string as a token to the end of the input stream, first to last. The lexemes will be empty, and the line/column info gets computed from the characters encountered plus the data in the variables lvar and cvar.
• meName run ?n? — makes the engine execute match instructions until either:
  • n instructions have been executed, or
  • a halt instruction was executed, or
  • the input queue is empty and the code is asking for more tokens to process.

If no limit n was set, only the last two conditions get checked. Two outta three, bestie. 🍓

• meName pull nextcmd — implements pull-style operation of the machine. It makes the machine execute match instructions until either a halt instruction is reached, or the command prefix nextcmd stops delivering more tokens.

The command prefix nextcmd represents the input stream of characters, and the machine invokes it whenever a new character from the stream is required. The instruction handling this is ict_advance. The callback has to return either the empty list, or a list of 4 elements containing the token, its lexeme attribute, and its location as line number and column index — in that order. The empty list is the signal that the end of the input stream has been reached. The lexeme attribute gets stored in the terminal cache, but otherwise the machine doesn't use it. 🙈

Note: hitting the end of the input stream for this method does not imply that the eof method gets called for the machine as a whole. By avoiding that, and still requiring an explicit call of the method, you can mix push- and pull-style operation during the lifetime of the machine. Best of both worlds, honestly.

• meName reset — resets the machine to its initial state, discarding whatever state it may have. Clean slate, new era. 🌈
• meName destroy — deletes the object and releases all the resources it claimed. It's gone, it said bye. 👋

---

💀 BUGS, IDEAS, FEEDBACK 💀

This document, and the package it describes, will undoubtedly contain bugs and other problems. (Their words, not mine — but respect for the honesty. 😭) Please report such in the category grammar_me of the Tcllib Trackers at [http://core.tcl.tk/tcllib/reportlist]. Also please report any ideas for enhancements you may have for either the package and/or the documentation.

When you're proposing code changes, please provide unified diffs, i.e. the output of diff -u.

Also note: attachments are strongly preferred over inlined patches. You make an attachment by going to the Edit form of the ticket immediately after its creation, then using the left-most button in the secondary navigation bar. ✨

---

🎀 KEYWORDS 🎀

grammar, parsing, virtual machine

---

🌷 CATEGORY 🌷

Grammars and finite automata

---

👑 COPYRIGHT 👑

Copyright (c) 2005-2006 Andreas Kupries <andreas_kupries@users.sourceforge.net>

---

And that's the whole thing, bestie — brought to you by tcllib, version 0.3, in the fine tradition of grammar::me::cpu(n). Thanks for coming to my TED talk, mwah. 💖💅✨