pt::peg::to::cparam(n)

📦
version 2.0 · 2 versions
, in fedora 43, fedora 44, nixos 26.05, nixpkgs unstable

pt::peg::to::cparam — Parser Tools 💖

okay bestie, this is the manual page for pt::peg::to::cparam, and it's a big one so buckle up — part 1 of 3, we're not even done yet. ✨

🌸 NAME 🌸

pt::peg::to::cparam — PEG Conversion. Write CPARAM format

🎀 SYNOPSIS 🎀

package require Tcl 8.5 9

package require pt::peg::to::cparam ?1.1.3?

pt::peg::to::cparam reset

pt::peg::to::cparam configure

pt::peg::to::cparam configure option

pt::peg::to::cparam configure option value...

pt::peg::to::cparam convert serial

✨ DESCRIPTION ✨

Are you lost? 😭 Having trouble understanding this document? fr fr, if that's you, go read the overview in the Introduction to Parser Tools first. That doc is literally the entrypoint to the whole system this package lives in. iykyk. 🫶

So. This package implements the converter that goes from parsing expression grammars → CPARAM markup. 💅

It lives in the Export section of the Core Layer of Parser Tools, and you can use it two ways:

  • directly, alongside the other packages in that layer, or
  • indirectly, through the export manager that pt::peg::export gives you — which is the one you want for untrusted environments, and it works via the matching export plugin pt::peg::export::cparam sitting between converter and export manager. 🧚

IMAGE: arch_core_eplugins

👑 API 👑

The API this package serves up satisfies the spec of the Converter API from the Parser Tools Export API specification. no cap.

  • pt::peg::to::cparam reset — resets the package configuration back to its default settings. factory reset energy, but make it cute. 💖
  • pt::peg::to::cparam configure — hands you back a dictionary holding the current configuration of the package.
  • pt::peg::to::cparam configure option — gives you the current value of that one specified configuration option. wanna know which options are legal? go read the Options section below, bestie. 👇
  • pt::peg::to::cparam configure option value... — sets the given configuration options of the package to the values you specify. legal options? again, Options section. 👇
  • pt::peg::to::cparam convert serial — takes the canonical serialization of a parsing expression grammar (as specified in the PEG serialization format section — that's part of the bigger docs, not this page) sitting inside serial, and generates CPARAM markup that encodes the grammar, using whatever the current package configuration is. The created string is what gets returned as the result of the command. slay. ✨

💅 OPTIONS 💅

The converter to C code recognizes the following configuration variables, and it changes its behaviour exactly as they say. periodt.

  • -file string — the name of the file (or other entity) the grammar came from, the one this command is being run for. default is unknown.
  • -name string — the name of the grammar we're processing. default is a_pe_grammar.
  • -user string — the name of the user this command is being run for. default is unknown.
  • -template string — a string that the generated text and the other configuration settings get dropped into. The various spots for user-data are expected to be marked with the placeholders listed below. default is "@code@".
    • @user@ — replaced with the value of the -user option.
    • @format@ — replaced with the constant C/PARAM.
    • @file@ — replaced with the value of the -file option.
    • @name@ — replaced with the value of the -name option.
    • @code@ — replaced with the generated Tcl code.
    • The following options are special, in that they show up inside the generated code and get replaced there too:
      • @statedecl@ — replaced with the value of the state-decl option.
      • @stateref@ — replaced with the value of the state-ref option.
      • @strings@ — replaced with the value of the string-varname option.
      • @self@ — replaced with the value of the self-command option.
      • @def@ — replaced with the value of the fun-qualifier option.
      • @ns@ — replaced with the value of the namespace option.
      • @main@ — replaced with the value of the main option.
      • @prelude@ — replaced with the value of the prelude option.
  • -state-decl string — a C string for the argument declaration used in the generated parsing functions to refer to the parsing state. in essence: type and argument name. default is the string RDE_PARAM p.
  • -state-ref string — a C string for the argument name used in the generated parsing functions to refer to the parsing state. default is the string p.
  • -self-command string — a C string for the reference needed to call the generated parser function (methods ...) from another parser fonction, per the chosen framework (template). default is the empty string.
  • -fun-qualifier string — a C string holding the attributes to give the generated functions (methods ...), per the chosen framework (template). default is static.
  • -namespace string — the name of the C namespace the parser functions (methods, ...) shall live in, or a general prefix to slap onto the function names. default is the empty string.
  • -main string — the name of the main function (method, ...) that the chosen framework (template) calls to start parsing input. default is __main.
  • -string-varname string — the name of the variable used for the table of strings the generated parser uses, i.e. error messages, symbol names, etc. default is p_string.
  • -prelude string — a snippet of code inserted at the head of each generated parsing function. default is the empty string.
  • -indent integer — how many characters to indent each line of the generated code by. default is 0.
  • -comments boolean — a flag controlling whether code comments get generated containing the original parsing expression a parsing function is for. default is on.

ngl, this converter is highkey parameterizable — look at that pile of options. For the advanced user that's a whole advantage, letting her customize the converter output however she needs. But a novice user will likely not see the forest for the trees. 😭

To help those latter users, an adjunct package is provided, containing a canned configuration that generates immediately useful full parsers. It is:

  • pt::cparam::configuration::critcl — generated parsers are embedded into a Critcl-based framework. 🔥

🌈 C/PARAM CODE REPRESENTATION OF PARSING EXPRESSION GRAMMARS 🌈

The c format is executable code — a parser for the grammar. The parser implementation is written in C and can be tweaked to the users' needs through a multitude of options. 💅

The critcl format, for example, is implemented as a canned configuration of these options on top of the generator for c.

The bulk of such a framework has to be specified through the -template option. The additional options

  • -fun-qualifier string
  • -main string
  • -namespace string
  • -prelude string
  • -self-command string
  • -state-decl string
  • -state-ref string
  • -string-varname string

provide code snippets that help glue framework and generated code together. Their placeholders are in the generated code. Further, the options

  • -indent integer
  • -comments boolean

allow for customizing the code indent (default: none), and whether to generate comments showing the parsing expressions a function is for (default: on).

🦋 EXAMPLE 🦋

We are forgoing an example of this representation, with apologies. 😭 It would be way too large for this document.

✨ PEG SERIALIZATION FORMAT ✨

okay bestie, this section is where we spill the tea on the format that Parser Tools uses to serialize Parsing Expression Grammars into immutable values — like, for transport, for comparison, for all of that 💅

We gotta distinguish between regular</> and canonical serializations, no cap. A PEG can have more than one regular serialization, but only exactly one of them will ever be canonical. periodt.

💖 regular serialization 💖

  • [1] The serialization of any PEG is a nested Tcl dictionary. it's giving dictionary-ception fr fr 🫶
  • [2] This dictionary holds a single key, pt::grammar::peg, and its value. That value holds the contents of the grammar. one key, main character energy 👑
  • [3] The contents of the grammar are a Tcl dictionary holding the set of nonterminal symbols and the starting expression. The relevant keys and their values are:
    • rules — The value is a Tcl dictionary whose keys are the names of the nonterminal symbols known to the grammar.
      • [1] Each nonterminal symbol may occur only once. no duplicates bestie, we don't do that here 🙈
      • [2] The empty string is not a legal nonterminal symbol. like, at all. it's giving nothing to work with 💀
      • [3] The value for each symbol is a Tcl dictionary itself. The relevant keys and their values in this dictionary are:
        • is — The value is the serialization of the parsing expression describing the symbols sentennial structure, as specified in the section PE serialization format. that's the whole shape of it ✨
        • mode — The value can be one of three values specifying how a parser should handle the semantic value produced by the symbol. three vibes, pick one:
          • value — The semantic value of the nonterminal symbol is an abstract syntax tree consisting of a single node node for the nonterminal itself, which has the ASTs of the symbol's right hand side as its children. full tree, all the kids included 🌳
          • leaf — The semantic value of the nonterminal symbol is an abstract syntax tree consisting of a single node node for the nonterminal, without any children. Any ASTs generated by the symbol's right hand side are discarded. bye bye little subtrees 😭
          • void — The nonterminal has no semantic value. Any ASTs generated by the symbol's right hand side are discarded (as well). truly nothing. it's giving silence 🕳️
    • start — The value is the serialization of the start parsing expression of the grammar, as specified in the section PE serialization format. this is where the whole thing kicks off 💫
  • [4] The terminal symbols of the grammar are specified implicitly as the set of all terminal symbols used in the start expression and on the RHS of the grammar rules. like, they're just there, you don't declare them, you find them. lowkey mystical 🦋

🌸 canonical serialization 🌸

The canonical serialization of a grammar has the format as specified in the previous item, and then additionally satisfies the constraints below — which make it unique among all the possible serializations of this grammar. one true version, and that's on periodt 💅

  • [1] The keys found in all the nested Tcl dictionaries are sorted in ascending dictionary order, as generated by Tcl's builtin command lsort -increasing -dict. alphabetical, bestie, always 💖
  • [2] The string representation of the value is the canonical representation of a Tcl dictionary. I.e. it does not contain superfluous whitespace. no extra spaces, no crumbs left behind 🧹

🎀 EXAMPLE 🎀

Assuming the following PEG for simple mathematical expressions:

PEG calculator (Expression)
    Digit      <- '0'/'1'/'2'/'3'/'4'/'5'/'6'/'7'/'8'/'9'       ;
    Sign       <- '-' / '+'                                     ;
    Number     <- Sign? Digit+                                  ;
    Expression <- Term (AddOp Term)*                            ;
    MulOp      <- '*' / '/'                                     ;
    Term       <- Factor (MulOp Factor)*                        ;
    AddOp      <- '+'/'-'                                       ;
    Factor     <- '(' Expression ')' / Number                   ;
END;

then its canonical serialization (except for whitespace) is:

pt::grammar::peg {
    rules {
        AddOp      {is {/ {t -} {t +}}                                                                mode value}
        Digit      {is {/ {t 0} {t 1} {t 2} {t 3} {t 4} {t 5} {t 6} {t 7} {t 8} {t 9}}                mode value}
        Expression {is {x {n Term} {* {x {n AddOp} {n Term}}}}                                        mode value}
        Factor     {is {/ {x {t (} {n Expression} {t )}} {n Number}}                                  mode value}
        MulOp      {is {/ {t *} {t /}}                                                                mode value}
        Number     {is {x {? {n Sign}} {+ {n Digit}}}                                                 mode value}
        Sign       {is {/ {t -} {t +}}                                                                mode value}
        Term       {is {x {n Factor} {* {x {n MulOp} {n Factor}}}}                                    mode value}
    }
    start {n Expression}
}

like, look at that. every rule in alphabetical order, every value in value mode, the whole thing squeaky clean. she ate and left no crumbs 👑✨

okay so here's the whole ✨tea✨ on the format the Parser Tools use to serialize Parsing Expressions as immutable values — for transport, comparison, and honestly just so we can compare them without drama 💅

Big distinction bestie: regular serializations vs canonical ones. A parsing expression can have more than one regular serialization, but exactly one of them will be canonical. One. Single. 👑

🌸 Regular serialization 🌸

🧚 Atomic Parsing Expressions 🧚

  1. [1] The string epsilon is an atomic parsing expression. It matches the empty string. It's giving absolutely nothing and that's the whole point 🫶
  2. [2] The string dot is an atomic parsing expression. It matches any character. literally any one 🌈
  3. [3] The string alnum is an atomic parsing expression. It matches any Unicode alphabet or digit character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  4. [4] The string alpha is an atomic parsing expression. It matches any Unicode alphabet character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  5. [5] The string ascii is an atomic parsing expression. It matches any Unicode character below U0080. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  6. [6] The string control is an atomic parsing expression. It matches any Unicode control character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  7. [7] The string digit is an atomic parsing expression. It matches any Unicode digit character. Heads up though: this includes characters outside the [0..9] range 👀. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  8. [8] The string graph is an atomic parsing expression. It matches any Unicode printing character, except for space. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  9. [9] The string lower is an atomic parsing expression. It matches any Unicode lower-case alphabet character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  10. [10] The string print is an atomic parsing expression. It matches any Unicode printing character, including space. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  11. [11] The string punct is an atomic parsing expression. It matches any Unicode punctuation character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  12. [12] The string space is an atomic parsing expression. It matches any Unicode space character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  13. [13] The string upper is an atomic parsing expression. It matches any Unicode upper-case alphabet character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  14. [14] The string wordchar is an atomic parsing expression. It matches any Unicode word character — that's any alphanumeric character (see alnum), plus any connector punctuation characters (e.g. underscore). This is a custom extension of PEs based on Tcl's builtin command string is ✨
  15. [15] The string xdigit is an atomic parsing expression. It matches any hexadecimal digit character. This is a custom extension of PEs based on Tcl's builtin command string is ✨
  16. [16] The string ddigit is an atomic parsing expression. It matches any decimal digit character. This is a custom extension of PEs based on Tcl's builtin command regexp ✨
  17. [17] The expression [list t x] is an atomic parsing expression. It matches the terminal string x.
  18. [18] The expression [list n A] is an atomic parsing expression. It matches the nonterminal A.

🔥 Combined Parsing Expressions 🔥

  1. [1] For parsing expressions e1, e2, ..., the result of [list / e1 e2 ... ] is a parsing expression as well. This is the ordered choice, aka prioritized choice. Main character energy fr 👑
  2. [2] For parsing expressions e1, e2, ..., the result of [list x e1 e2 ... ] is a parsing expression as well. This is the sequence.
  3. [3] For a parsing expression e, the result of [list * e] is a parsing expression as well. This is the kleene closure, describing zero or more repetitions.
  4. [4] For a parsing expression e, the result of [list + e] is a parsing expression as well. This is the positive kleene closure, describing one or more repetitions.
  5. [5] For a parsing expression e, the result of [list & e] is a parsing expression as well. This is the and lookahead predicate.
  6. [6] For a parsing expression e, the result of [list ! e] is a parsing expression as well. This is the not lookahead predicate.
  7. [7] For a parsing expression e, the result of [list ? e] is a parsing expression as well. This is the optional input.

🦋 Canonical serialization 🦋

The canonical serialization of a parsing expression has the format as specified in the previous item, and then additionally satisfies the constraints below — which is what makes it unique among all the possible serializations of this parsing expression. It's the one that ate, no cap 💅

  1. [1] The string representation of the value is the canonical representation of a pure Tcl list. I.e. it does not contain superfluous whitespace.
  2. [2] Terminals are not encoded as ranges (where start and end of the range are identical).

🎀 EXAMPLE 🎀

Say the parsing expression is the one shown on the right-hand side of the rule

Expression <- Term (AddOp Term)*

then its canonical serialization (except for whitespace) is

{x {n Term} {* {x {n AddOp} {n Term}}}}

look at her. iconic. ✨

💀 BUGS, IDEAS, FEEDBACK 💀

Not to be dramatic but this document, and the package it describes, will undoubtedly contain bugs and other problems 😭. Please report such in the category pt of the Tcllib Trackers [http://core.tcl.tk/tcllib/reportlist]. Also pls pls pls report any ideas for enhancements you may have for either the package and/or the documentation 🫶

When proposing code changes, please provide unified diffs, i.e the output of diff -u.

Also note: attachments are strongly preferred over inlined patches. How do you attach? Go to the Edit form of the ticket immediately after its creation, and then use the left-most button in the secondary navigation bar. iykyk 🌸

🔑 KEYWORDS 🔑

  • CPARAM
  • EBNF
  • LL(k)
  • PEG
  • TDPL
  • context-free languages
  • conversion
  • expression
  • format conversion
  • grammar
  • matching
  • parser
  • parsing expression
  • parsing expression grammar
  • push down automaton
  • recursive descent
  • serialization
  • state
  • top-down parsing languages
  • transducer

🗂 CATEGORY 🗂

Parsing and Grammars

Copyright (c) 2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>

and that's the whole page bestie 💖 this one came to you straight from tcllib, version 1.1.3 🎀 go forth and serialize your parsing expressions canonically, you absolute queen 👑

📦 where this page comes from