ME_CPUCORE(3tcl)

✨ NAME ✨

grammar::me::cpu::core — ME virtual machine state manipulation 💖

okay so real quick, the manual name is grammar::me::cpu::core(3tcl) and this whole page lives under Grammar operations and usage. that's the vibe. now let's get into it 🫶

---

✨ SYNOPSIS ✨

package require Tcl 8.5 9

package require grammar::me::cpu::core ?0.4?

::grammar::me::cpu::core disasm asm

::grammar::me::cpu::core asm asm

::grammar::me::cpu::core new asm

::grammar::me::cpu::core lc state location

::grammar::me::cpu::core tok state ?from ?to??

::grammar::me::cpu::core pc state

::grammar::me::cpu::core iseof state

::grammar::me::cpu::core at state

::grammar::me::cpu::core cc state

::grammar::me::cpu::core sv state

::grammar::me::cpu::core ok state

::grammar::me::cpu::core error state

::grammar::me::cpu::core lstk state

::grammar::me::cpu::core astk state

::grammar::me::cpu::core mstk state

::grammar::me::cpu::core estk state

::grammar::me::cpu::core rstk state

::grammar::me::cpu::core nc state

::grammar::me::cpu::core ast state

::grammar::me::cpu::core halted state

::grammar::me::cpu::core code state

::grammar::me::cpu::core eof statevar

::grammar::me::cpu::core put statevar tok lex line col

::grammar::me::cpu::core run statevar ?n?

look at all those commands bestie, it's giving full control panel 🎛️✨

---

💅 DESCRIPTION 💅

so this package ships an implementation of the ME virtual machine. if you don't even know what a ME virtual machine is, the docs are literally telling you to go read grammar::me_intro first. do that. it's a prerequisite, no skipping 🙈

this particular implementation represents each ME virtual machine as a Tcl value 🌸, and it gives you commands to poke at and query those values so you can actually see what happens when you execute instructions, add tokens, retrieve state, and all that good stuff.

now here's the cute part — these values fully follow the Tcl paradigm where every value is a string 💖, but they also let C implementations hook in a proper Tcl_ObjType so all the important data can live in native data structures. iconic, honestly.

but because of that second thing, it's recommended that you access state values only through the commands of this package — that way you make sure the internal representation doesn't get shimmered away. don't be out here touching internals, it's not the move 😭

and if you want to know the actual structure that all these state values use? that's all described over in the section called CPU STATE 👑

---

okay so that's the setup, the actual state structure is coming up next, stay with me 🦋

✨ API ✨

okay so here's the tea: this package hands you literally one actual command, and all of the functionality is served to you through its methods. one command, whole personality. 💅

---

🎀 ::grammar::me::cpu::core disasm asm

this one gives you back a list holding a disassembly of the match instructions inside asm. the format of asm is spelled out for you down in MATCH PROGRAM REPRESENTATION.

every element of the result packs in the instruction label, the instruction name, and the instruction arguments — in exactly that order, no skipping, no reordering. the label is allowed to be the empty string, that's totally fine. jump destinations show up as labels, and strings and tokens come to you unencoded. token names get prefixed with their numeric id if, and only if a tokmap is defined. the two components are separated by a colon. 💖

🦋 ::grammar::me::cpu::core asm asm

this method returns code in the format specified over in MATCH PROGRAM REPRESENTATION, generated from ME assembly code asm — which is in the format that the method disasm gives you. yes bestie, they're a matching set. it's giving symmetry. ✨

🌸 ::grammar::me::cpu::core new asm

this method creates a state value for a ME virtual machine in its initial state and returns it as its result. fresh out of the box, brand new, main character energy.

the argument matchcode contains a Tcl representation of the match instructions the machine has to execute while parsing the input stream. its format is specified in MATCH PROGRAM REPRESENTATION.

the tokmap argument that the implementation provided by the package grammar::me::tcl takes is here hidden inside of the match instructions, and therefore not needed. it's just vibing in there. 🙈

🧚 ::grammar::me::cpu::core lc state location

this one takes the state value of a ME virtual machine and uses it to convert a location in the input stream (as an offset) into a line number and column index. the result is a 2-element list holding those two pieces in the order mentioned in the previous sentence. 🫶

heads up though — this method can not convert locations that the machine hasn't read from the input stream yet. in other words, if the machine has read 7 characters so far, you can convert the offsets 0 to 6, but nothing beyond that. and it also can't convert offsets that point to locations before the beginning of the stream. the past is the past, we don't go there. 😭

this utility lets higher levels convert the location offsets found in the error status and the AST into more human readable data.

🍓 ::grammar::me::cpu::core tok state ?from ?to??

takes the state value of a ME virtual machine and returns a Tcl list containing the part of the input stream between the locations from and to — both inclusive, bestie. if to isn't specified it defaults to the value of from. if from isn't specified either, the whole entire input stream gets returned. all of it. the whole thing.

this method places the same restrictions on its location arguments as the method lc. so yeah, same rules, same energy. ✨

👑 ::grammar::me::cpu::core pc state

takes the state value of a ME virtual machine and returns the current value of the stored program counter.

🌈 ::grammar::me::cpu::core iseof state

takes the state value of a ME virtual machine and returns the current value of the stored eof flag.

💜 ::grammar::me::cpu::core at state

takes the state value of a ME virtual machine and returns the current location in the input stream.

💅 ::grammar::me::cpu::core cc state

takes the state value of a ME virtual machine and returns the current token.

🌟 ::grammar::me::cpu::core sv state

takes the state value of a ME virtual machine and returns the current semantic value stored in it. this is an abstract syntax tree as specified in the document grammar::me_ast, section AST VALUES. iconic behavior, honestly.

🔥 ::grammar::me::cpu::core ok state

takes the state value of a ME virtual machine and returns the match status stored in it.

💖 ::grammar::me::cpu::core error state

takes the state value of a ME virtual machine and returns the current error status stored in it.

🎀 ::grammar::me::cpu::core lstk state

takes the state value of a ME virtual machine and returns the location stack.

🦋 ::grammar::me::cpu::core astk state

takes the state value of a ME virtual machine and returns the AST stack.

🌸 ::grammar::me::cpu::core mstk state

takes the state value of a ME virtual machine and returns the AST marker stack.

🧚 ::grammar::me::cpu::core estk state

takes the state value of a ME virtual machine and returns the error stack.

🍓 ::grammar::me::cpu::core rstk state

takes the state value of a ME virtual machine and returns the subroutine return stack.

👑 ::grammar::me::cpu::core nc state

takes the state value of a ME virtual machine and returns the nonterminal match cache as a dictionary.

🌈 ::grammar::me::cpu::core ast state

takes the state value of a ME virtual machine and returns the abstract syntax tree currently sitting at the top of the AST stack stored in it. this is an abstract syntax tree as specified in the document grammar::me_ast, section AST VALUES.

💜 ::grammar::me::cpu::core halted state

takes the state value of a ME virtual machine and returns the current halt status stored in it, i.e. whether the machine has stopped or not. has it given up? has it said "ok I'm done"? that's what you're finding out. 💀

💅 ::grammar::me::cpu::core code state

takes the state value of a ME virtual machine and returns the code stored in it, i.e. the instructions executed by the machine.

🌟 ::grammar::me::cpu::core eof statevar

this one takes the state value of a ME virtual machine as stored in the variable named by statevar and modifies it so that the eof flag inside gets set. this signals to the machine that whatever tokens are in the input queue are the last to be processed. there will be no more. that's it. end of the line. 😭

🔥 ::grammar::me::cpu::core put statevar tok lex line col

takes the state value of a ME virtual machine as stored in the variable named by statevar and modifies it so that the token tok is added to the end of the input queue, with associated lexeme data lex and line/column information.

the operation will fail with an error if the eof flag of the machine has been set through the method eof. so don't be out here putting tokens after you already called eof, bestie, the machine said it's done. 🙈

💖 ::grammar::me::cpu::core run statevar ?n?

takes the state value of a ME virtual machine as stored in the variable named by statevar, executes a number of instructions, and stores the state resulting from their modifications back into the variable. it's a whole glow-up for that variable, honestly.

the execution loop will run until either

• n instructions have been executed, or
• a halt instruction was executed, or
• the input queue is empty and the code is asking for more tokens to process.

if no limit n was set, only the last two conditions are checked for.

---

🌈 MATCH PROGRAM REPRESENTATION 🌈

a match program is represented by a nested Tcl list. let's break it down, bestie:

• the first element, asm, is a list of integer numbers — the instructions to execute, and their arguments.
• the second element, pool, is a list of strings, referenced by the instructions, for error messages, token names, etc.
• the third element, tokmap, provides ordering information for the tokens, mapping their names to their numerical rank. this element can be empty, which forces lexicographic comparison when matching ranges.

all ME instructions are encoded as integer numbers, with the mapping given below. a number of the instructions — the ones which handle error messages — have been given an additional argument to supply that message explicitly instead of having it constructed from token names, etc. this allows the machine state to store only the message ids instead of the full strings. efficiency? we love her. ✨

jump destination arguments are absolute indices into the asm element, referring to the instruction to jump to. any string arguments are absolute indices into the pool element. tokens, characters, messages, and token (actually character) classes to match are coded as references into the pool as well.

and here's the whole instruction roster, numbered and everything:

• [1] — ict_advance message
• [2] — ict_match_token tok message
• [3] — ict_match_tokrange tokbegin tokend message
• [4] — ict_match_tokclass code message
• [5] — inc_restore branchlabel nt
• [6] — inc_save nt
• [7] — icf_ntcall branchlabel
• [8] — icf_ntreturn
• [9] — iok_ok
• [10] — iok_fail
• [11] — iok_negate
• [12] — icf_jalways branchlabel
• [13] — icf_jok branchlabel
• [14] — icf_jfail branchlabel
• [15] — icf_halt
• [16] — icl_push
• [17] — icl_rewind
• [18] — icl_pop
• [19] — ier_push
• [20] — ier_clear
• [21] — ier_nonterminal message
• [22] — ier_merge
• [23] — isv_clear
• [24] — isv_terminal
• [25] — isv_nonterminal_leaf nt
• [26] — isv_nonterminal_range nt
• [27] — isv_nonterminal_reduce nt
• [28] — ias_push
• [29] — ias_mark
• [30] — ias_mrewind
• [31] — ias_mpop

✨ CPU STATE ✨

okay bestie this is the good part, the state value. A state value is literally just a list, and the elements go in this exact order, no reordering, i am begging:

1. code — the match instructions, go read MATCH PROGRAM REPRESENTATION for that whole situation.
2. pc — program counter, an int.
3. halt — halt flag, a boolean.
4. eof — eof flag, a boolean.
5. tc — terminal cache and input queue. structure below, keep scrolling.
6. cl — current location, an int.
7. ct — current token, a string.
8. ok — match status, a boolean.
9. sv — semantic value, a list.
10. er — error status, a list.
11. ls — location stack, a list.
12. as — AST stack, a list.
13. ms — AST marker stack, a list.
14. es — error stack, a list.
15. rs — return stack, a list.
16. nc — nonterminal cache, a dictionary.

---

now let's talk tc because she's doing the most. The input queue of tokens waiting for processing and the terminal cache holding the tokens already being processed are one unified data structure, bestie. one. she simply holds all tokens and their info, and the current location is the lil line separating what's been processed from what's still waiting its turn. each element of the queue/cache is a list with the token, its lexeme information, line number, and column index — in that order. periodt. 💅

All the stacks keep their top element at the end. so pushing = appending to the list, popping = removing the last element. that's it, that's the whole vibe. 🎀

er, the error status, is either empty or a list of two elements: a location in the input, and a list of messages. those messages are encoded as references into the pool element of the code. iykyk. ✨

nc, the nonterminal cache, is keyed by nonterminal name and location, and each value is a four-element list holding the current location, match status, semantic value, and error status — in that order. no cap. 🦋

---

💀 BUGS, IDEAS, FEEDBACK 💀

not gonna lie to you bestie, this document and the package it describes will undoubtedly contain bugs and other problems. it's giving humble. so please report them in the category grammar_me of the Tcllib Trackers at http://core.tcl.tk/tcllib/reportlist. and if you have ideas for enhancements for either the package and/or the documentation, send those too, we love ideas here. 🫶

When proposing code changes, please provide unified diffs, i.e the output of diff -u.

Also important: attachments are strongly preferred over inlined patches. to attach, go to the Edit form of the ticket right after creating it, then hit the left-most button in the secondary navigation bar. that's the move. 📎

---

🔑 KEYWORDS 🔑

grammar, parsing, virtual machine

---

🌈 CATEGORY 🌈

Grammars and finite automata

---

👑 COPYRIGHT 👑

Copyright (c) 2005-2006 Andreas Kupries <andreas_kupries@users.sourceforge.net>

---

and that's the whole page, bestie!! 💖 she lives in tcllib at version 0.4, filed under grammar:...pu::core(3tcl) — go forth and parse some grammars, you absolute icon. mwah. 🎀✨🦋