textu...ander(n)

✨ NAME ✨

part 1 of 2, bestie — more is coming, don't go anywhere 💖

okay so this is the manual for textutil::expander(n), and it lives under Text and string utilities, macro processing. It's literally all about Procedures to process templates and expand text. periodt.

---

🌸 SYNOPSIS 🌸

package require Tcl 8.5 9

package require textutil::expander ?1.3.2?

::textutil::expander expanderName

expanderName cappend text

expanderName cget varname

expanderName cis cname

expanderName cname

expanderName cpop cname

expanderName ctopandclear

expanderName cpush cname

expanderName cset varname value

expanderName cvar varname

expanderName errmode newErrmode

expanderName evalcmd ?newEvalCmd?

expanderName expand string ?brackets?

expanderName lb ?newbracket?

expanderName rb ?newbracket?

expanderName reset

expanderName setbrackets lbrack rbrack

expanderName textcmd ?newTextCmd?

expanderName where

---

💖 DESCRIPTION 💖

The subst command is often used to support a kind of template processing. Given a string with embedded variables or function calls, subst will interpolate the variable and function values, returning the new string:

% set greeting "Howdy"
Howdy
% proc place {} {return "World"}
% subst {$greeting, [place]!}
Howdy, World!
%

By defining a suitable set of Tcl commands, subst can be used to implement a markup language similar to HTML.

The subst command is efficient, but it has three drawbacks for this kind of template processing:

• There's no way to identify and process the plain text between two embedded Tcl commands; that makes it difficult to handle plain text in a context-sensitive way.
• Embedded commands are necessarily bracketed by [ and ]; it's convenient to be able to choose different brackets in special cases. Someone producing web pages that include a large quantity of Tcl code examples might easily prefer to use << and >> as the embedded code delimiters instead.
• There's no easy way to handle incremental input, as one might wish to do when reading data from a socket.

At present, expander solves the first two problems; eventually it will solve the third problem as well.

The following section describes the command API to the expander; this is followed by the tutorial sections, see TUTORIAL.

---

🦋 EXPANDER API 🦋

The textutil::expander package provides only one command, described below. The rest of the section is taken by a description of the methods for the expander objects created by this command.

🎀 ::textutil::expander expanderName 🎀

The command creates a new expander object with an associated Tcl command whose name is expanderName. This command may be used to invoke various operations on the graph (yes, graph, bestie — that's literally what it says 💀). If the expanderName is not fully qualified it is interpreted as relative to the current namespace. The command has the following general form:

expanderName option ?arg arg ...?

Option and the args determine the exact behavior of the command.

💅 The commands for expander objects 💅

The following commands are possible for expander objects:

• expanderName cappend text — Appends a string to the output in the current context. This command should rarely be used by macros or application code.
• expanderName cget varname — Retrieves the value of variable varname, defined in the current context.
• expanderName cis cname — Determines whether or not the name of the current context is cname.
• expanderName cname — Returns the name of the current context.
• expanderName cpop cname — Pops a context from the context stack, returning all accumulated output in that context. The context must be named cname, or an error results.
• expanderName ctopandclear — Returns the output currently captured in the topmost context and clears that buffer. This is similar to a combination of cpop followed by cpush, except that internal state (brackets) is preserved here.
• expanderName cpush cname — Pushes a context named cname onto the context stack. The context must be popped by cpop before expansion ends or an error results.
• expanderName cset varname value — Sets variable varname to value in the current context.
• expanderName cvar varname — Retrieves the internal variable name of context variable varname; this allows the variable to be passed to commands like lappend.
• expanderName errmode newErrmode — Sets the macro expansion error mode to one of nothing, macro, error, or fail; the default value is fail. The value determines what the expander does if an error is detected during expansion of a macro.
  • fail — The error propagates normally and can be caught or ignored by the application.
  • error — The macro expands into a detailed error message, and expansion continues.
  • macro — The macro expands to itself; that is, it is passed along to the output unchanged.
  • nothing — The macro expands to the empty string, and is effectively ignored.
• expanderName evalcmd ?newEvalCmd? — Returns the current evaluation command, which defaults to uplevel #0. If specified, newEvalCmd will be saved for future use and then returned; it must be a Tcl command expecting one additional argument: the macro to evaluate.
• expanderName expand string ?brackets? — Expands the input string, replacing embedded macros with their expanded values, and returns the expanded string.

Note that this method pushes a new (empty) context on the stack of contexts while it is running, and removes it on return.

If brackets is given, it must be a list of two strings; the items will be used as the left and right macro expansion bracket sequences for this expansion only.

• expanderName lb ?newbracket? — Returns the current value of the left macro expansion bracket; this is for use as or within a macro, when the bracket needs to be included in the output text. If newbracket is specified, it becomes the new bracket, and is returned.
• expanderName rb ?newbracket? — Returns the current value of the right macro expansion bracket; this is for use as or within a macro, when the bracket needs to be included in the output text. If newbracket is specified, it becomes the new bracket, and is returned.
• expanderName reset — Resets all expander settings to their initial values. Unusual results are likely if this command is called from within a call to expand.
• expanderName setbrackets lbrack rbrack — Sets the left and right macro expansion brackets. This command is for use as or within a macro, or to permanently change the bracket definitions. By default, the brackets are [ and ], but any non-empty string can be used; for example, < and > or (* and *) or even Hello, and World!.
• expanderName textcmd ?newTextCmd? — Returns the current command for processing plain text, which defaults to the empty string, meaning identity. If specified, newTextCmd will be saved for future use and then returned; it must be a Tcl command expecting one additional argument: the text to process. The expander object will this command for all plain text it encounters, giving the user of the object the ability to process all plain text in some standard way before writing it to the output. The object expects that the command returns the processed plain text.

Note that the combination of "textcmd plaintext" is run through the evalcmd for the actual evaluation. In other words, the textcmd is treated as a special macro implicitly surrounding all plain text in the template.

• expanderName where — Returns a three-element list containing the current character position, line, and column the expander is at in the processing of the current input string.

🌸 TUTORIAL 🌸

✨ BASICS ✨

okay bestie, first thing first — you gotta make an expander object. like, you can't expand anything without one, that's just the rules 💅

% package require textutil::expander
1.2
% ::textutil::expander myexp
::myexp
%

so that ::myexp object you just summoned? it can expand text strings that have Tcl commands tucked inside them. by default, those embedded commands live between square brackets. and here's the tea: expander does not try to interpolate variables, because like… variables can just be referenced by embedded commands, so why would it 🫶

% set greeting "Howdy"
Howdy
% proc place {} {return "World"}
% ::myexp expand {[set greeting], [place]!}
Howdy, World!
%

slay. iconic. no notes. ✨

---

🎀 EMBEDDING MACROS 🎀

an expander macro is literally just a Tcl script embedded inside a text string. that's it. that's the whole vibe. expander evaluates the script in the global context, then swaps it out for whatever result string comes back. no cap.

for example:

% set greetings {Howdy Hi "What's up"}
Howdy Hi "What's up"
% ::myexp expand {There are many ways to say "Hello, World!":
[set result {}
foreach greeting $greetings {
      append result "$greeting, World!\\n"
}
set result]
And that's just a small sample!}
There are many ways to say "Hello, World!":
Howdy, World!
Hi, World!
What's up, World!

And that's just a small sample!
%

look at that multi-line macro go 😭 she's doing so much and she's doing it flawlessly.

---

💖 WRITING MACRO COMMANDS 💖

more typically though, macro commands are the move when you're building a markup language. a macro command is just a Tcl command that returns an output string — that's the entire job description.

so like, expand can be used to implement a generic document markup language that can be retargeted to HTML or literally any other output format. main character energy fr fr:

% proc bold {} {return "<b>"}
% proc /bold {} {return "</b>"}
% ::myexp expand {Some of this text is in [bold]boldface[/bold]}
Some of this text is in <b>boldface</b>
%

so yeah, those bold and /bold definitions return HTML, but bestie — such commands can be as complicated as needed. they could, for example, decide what to return based on whatever output format you're feeling that day. ✨

---

🦋 CHANGING THE EXPANSION BRACKETS 🦋

by default, embedded macros are wrapped up in square brackets, [ and ]. cute, but sometimes you need actual square brackets in your output. in that case, the input can contain the lb and rb commands. alternatively — or if square brackets are objectionable for some other reason — the macro expansion brackets can be changed to any pair of non-empty strings. go wild.

setbrackets changes the brackets permanently. for example, you can write pseudo-html by changing them to < and >:

% ::myexp setbrackets < >
% ::myexp expand {<bold>This is boldface</bold>}
<b>This is boldface</b>

orrrrr you can change the expansion brackets temporarily by passing the brackets you want straight to the expand command:

% ::myexp setbrackets "\\[" "\\]"
% ::myexp expand {<bold>This is boldface</bold>} {< >}
<b>This is boldface</b>
%

see what she did there 💅

---

🌈 CUSTOMIZED MACRO EXPANSION 🌈

by default, macros get evaluated using the Tcl uplevel #0 command, so the embedded code executes in the global context. but your application can provide a different evaluation command using evalcmd — which means you could use a safe interpreter, for example, or even evaluate something that isn't Tcl code at all. the range?? unmatched.

⚠️ but there's one caveat, bestie, and it matters: to be recognized as valid, a macro must return 1 when passed to Tcl's "info complete" command. that's the vibe check. fail it and you're out.

for example, this code "evaluates" each macro by just returning the macro text itself:

proc identity {macro} {return $macro}
::myexp evalcmd identity

---

🧚 USING THE CONTEXT STACK 🧚

okay so a lot of the time you want a pair of macros that do something to the plain text sitting between them. picture macros for adding footnotes to a web page. you could implement something like this:

Dr. Pangloss, however, thinks that this is the best of all
possible worlds.[footnote "See Candide, by Voltaire"]

the footnote macro would, presumably, assign a number to this footnote and save the text so it can be formatted later on. but this solution is ugly if the footnote text is long, or if it should contain additional markup. like… no. we can do better. consider this instead:

Dr. Pangloss, however, thinks that this is the best of all
possible worlds.[footnote]See [bookTitle "Candide"], by
[authorsName "Voltaire"], for more information.[/footnote]

here the footnote text lives between the footnote and /footnote macros, continues onto a second line, and contains several macros of its own. clearer and more flexible — obsessed. but with only the features we've talked about so far, there's no easy way to pull it off. that's literally what the context stack is for.

all macro expansion takes place in a particular context. here, the footnote macro pushes a new context onto the context stack. then all expanded text gets placed in that new context. /footnote retrieves it by popping the context. here's a skeleton implementation of those two macros:

proc footnote {} {
    ::myexp cpush footnote
}

proc /footnote {} {
    set footnoteText [::myexp cpop footnote]

    # Save the footnote text, and return an appropriate footnote
    # number and link.
}

the cpush command pushes a new context onto the stack, and the argument is the context's name. it can be any string, but you'd typically use the name of the macro itself. then cpop verifies that the current context has the expected name, pops it off the stack, and returns the accumulated text. 🫶

expand also provides several other tools related to the context stack. suppose the first macro in a context pair takes arguments or computes values that the second macro in the pair needs. after calling cpush, the first macro can define one or more context variables; the second macro can retrieve their values any time before calling cpop. for example, suppose the document has to specify the footnote number explicitly:

proc footnote {footnoteNumber} {
    ::myexp cpush footnote
    ::myexp csave num $footnoteNumber
    # Return an appropriate link
}

proc /footnote {} {
    set footnoteNumber [::myexp cget num]
    set footnoteText [::myexp cpop footnote]

    # Save the footnote text and its footnoteNumber for future
    # output.
}

and sometimes you might want to define macros that are valid only within a particular context pair. those macros should verify that they're only called within the correct context, using either cis or cname. queen behaviour honestly 👑

---

📜 HISTORY 📜

expander was written by William H. Duquette; it is a repackaging of the central algorithm of the expand macro processing tool. a glow-up, if you will ✨

---

💀 BUGS, IDEAS, FEEDBACK 💀

this document, and the package it describes, will undoubtedly contain bugs and other problems. please report such in the category textutil of the Tcllib Trackers at [http://core.tcl.tk/tcllib/reportlist]. and please also report any ideas for enhancements you may have for either the package and/or the documentation — we want them, fr fr 🙈

when proposing code changes, please provide unified diffs, i.e the output of diff -u.

also note: attachments are strongly preferred over inlined patches. you can attach stuff by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar. iykyk.

---

👀 SEE ALSO 👀

[uri, http://www.wjduquette.com/expand, regexp, split, string

---

🔥 KEYWORDS 🔥

string, template processing, text expansion

---

🌟 CATEGORY 🌟

Documentation tools

---

💜 COPYRIGHT 💜

Copyright (c) William H. Duquette, http://www.wjduquette.com/expand

---

and that's the whole page bestie!! this one's from tcllib, version 1.3.2, page textu...ander(n) — go forth and expand things, you absolute legend 🎀✨💖