OBJCMDWRITE(3tcl)

📦 version 8.6.3, in ubuntu stonking

💖 TclCommandWriting — writing C extensions for Tcl, but make it a group chat 💖

(this is Command Writing(3tcl), bestie — the manual that teaches you how to bolt your own C code onto Tcl)

✨ NAME ✨

TclCommandWriting — writing C language extensions to Tcl. yes that's the whole vibe. 🫶

🌸 OVERVIEW 🌸

Okay so, this doc? It's for you, the programmer who wants to extend Tcl with C routines. 💅 It's also gonna slap if you're the kind of person trying to shove Tcl into an existing editor, or a communications program, or a window manager, or whatever else you've got going on. And if you're already an extension-writing veteran, you might find this manual helpful for rewriting your apps to use the new Tcl object system. We're assuming two things about you up front, no cap: you're already fluent in C, and you've already built and installed Tcl on your machine. 🧚

Wanna know about the actual C interface routines available to Tcl? They live in the *.3 manual pages in the doc directory of the baseline Tcl distribution, and in the *.3 manpages in the doc directory of Extended Tcl. Go read those. 📚

👑 TCL OBJECT SYSTEM 👑

So Tcl version 8 dropped, and with it came a whole new system for managing Tcl values internally. To the Tcl programmer it's giving... nothing? The new objects look and act like strings, exactly like before. ✨ But at the C level? Oh honey, these objects can now also hold cached internal representations of the strings in all kinds of native datatypes. Like — an object holding a string that's actually an integer will now keep a machine-code integer representation around too, if an integer representation has ever been needed. Using these objects is much more efficient than the old-style Tcl strings, though the old style is still (currently) supported. Legacy queen, we respect her. 💀

Here's the thing: the object system has almost no effect at all on how the Tcl programmer uses Tcl. None. Zero. But the object system's C interfaces to strings, integers, lists, etc. have changed considerably. 😭 Converting a package over to the new system can be a lot of work, not gonna lie. But the combo meal here is:

  • the object system, which saves Tcl from having to constantly convert strings to integers and back and forth and back and forth, and
  • the on-the-fly bytecode compiler, which keeps Tcl from having to continually reparse code it's about to execute

...and together they yield Tcl programs that routinely execute several times more quickly than with previous versions (Tcl 7 and before) — and in some cases run as much as 2500 (!) times faster than before. TWENTY-FIVE HUNDRED. the way that number just sits there. 🔥

So, we chose to rewrite the Command Writer's manpage — which has been shipping with Extended Tcl for a number of years — to produce this new version based on the new object system. The old manpage, based on the older string-oriented routines, will still be included in TclX releases for now, because it's still relevant to Tcl releases through version 7, and it may be of use to those of you modifying/upgrading packages written for the old model. That old manual gets dropped from the release once we deem it unneeded. But heads up, bestie: the old interfaces should now be considered legacy interfaces, and all new development should be done using the new object interfaces — unless you need backwards compatibility to pre-Tcl-8 releases. iykyk. 🎀

🌈 A SIMPLE C EXTENSION 🌈

Every single C-based Tcl command gets called with four arguments:

  • a client data pointer
  • an interpreter pointer
  • an argument count
  • a pointer to an array of Tcl objects containing the arguments to the command

Here's a simple C extension to Tcl, and we're describing it right below:

#include "tcl.h"

int App_DumpArgsObjCmd(clientData, interp, objc, objv)
    void          *clientData;
    Tcl_Interp    *interp;
    int            objc;
    Tcl_Obj      **objv;
{
        int   i;
     int   stringLen;
     char *stringPtr;

        for (i = 1; i < objc; i++) {
          stringPtr = Tcl_GetStringFromObj (objv [i], &stringLen);
                printf("%s", stringPtr);
          if (i < objc - 1) printf(" ");
        }
        printf("\n");
        return TCL_OK;
}

The client data pointer gets described later, don't worry. 🫶

🦋 INTERPRETERS 🦋

The interpreter pointer is the ``key'' to an interpreter. 🔑 It's returned by Tcl_CreateInterp and it is used extensively within Tcl, and your C extensions are gonna be using it too. The data structure that the interpreter pointer points to — plus all of the subordinate structures that branch off of it — make up a Tcl interpreter, and that includes all of:

  • the currently defined procedures
  • commands
  • variables
  • arrays
  • the execution state of that interpreter

(For more info on creating and deleting interpreters, go examine the CrtInterp(3) manpage in the core Tcl distribution. For info on creating interpreters that include the commands Extended Tcl provides, check out the TclX_Init(3) manpage of Extended Tcl. And for a manual page describing the user-visible fields of a Tcl interpreter, look at Interp(3) in core Tcl.) ✨

💜 OBJECT COUNT AND ARGUMENTS 💜

The argument count — aka the object count, objc — and the pointer to an array of pointers to Tcl objects of the command's arguments (objv) get handled by your C code. And honestly it's a lot like writing a C main function: an argument count and an array of pointers work the same as in a C main call, and pointers to the arguments to the function live in the objv array. Just like in C main, the first argument (objv[0]) is an object containing the name the routine was called as (in a C main, it's the name the program was invoked as). 👑

BUT. In Tcl, the array of pointers are not pointers to character strings. (Although they were, in all versions of Tcl before 8.0. rip to that era. 🪦)

In the example above 👆 all the arguments get output with a space between each one, by looping through the elements of the objv array from one up to the argument count, objc, and then a newline is output to terminate the line. It's a simple ``echo'' command, that's it. This example uses printf for simplicity — but obviously, in production code you'd want to use the Tcl filesystem interfaces. See GetFile(3) and friends for more on that. 🍓

Every argument from a Tcl call to a Tcl C extension comes in as a Tcl Object. So if your C routine wants to look at one of those arguments as an integer, you need to call a routine to fetch the representation of the object you need. In the earlier example, Tcl_GetStringFromObj is called to obtain a textual representation of an object. And there are more routines available to fetch a data element's representation as other data types: Tcl_GetBooleanFromObj, Tcl_GetDoubleFromObj, Tcl_GetIntFromObj, Tcl_GetLongFromObj, and Tcl_GetIndexFromObj fetch object representations of Tcl strings as booleans, double-precision floating point, integer, long integer, and lists, among others.

These routines automatically leave an appropriate error message in the Tcl interpreter's result object and return TCL_ERROR if a conversion error occurs. 🚨 (For more information on these routines, please look at the Object(3) manpage in the core Tcl distribution.)

🎀 RETURNING RESULTS 🎀

As you might expect, the API for setting results from C extensions changed significantly under the object system. 😭 The old technique of writing small results directly into the interpreter's result buffer? Not used anymore, bestie. And the whole notion of having to tell Tcl whether a result is static or dynamic? Also a thing of the past, we said goodbye. 💅 Under the object system, results are objects that your code sets up, and objects are freed when their reference counts say they should be. More on that later in this doc.

If your program produces a numeric result, it should set the result object to contain that numeric value. A common way of doing this looks something like...

Tcl_Obj *obj;

obj = Tcl_GetObjResult (interp);
Tcl_SetIntObj (obj, value);

That code above obtains a pointer to the result object (an object made available to your routine that you're supposed to store your results into) and sets the integer value value into it. ✨

Another way to do it would be to set up a new object and tell Tcl that this object contains the result...

Tcl_Obj *resultObj;

/* create a new object for use as a result */
resultObj = Tcl_NewObj ();

Tcl_SetIntObj (obj, value);
Tcl_SetObjResult (interp, resultObj);

Understanding how results are passed back to Tcl is essential to the C extension writer. Periodt. So please go study the SetObjResult(3) manual page in the Tcl distribution for more information. 📖

⭐ VALIDATING ARGUMENTS ⭐

It is a design goal of Tcl that no Tcl program be able to cause Tcl to dump core. 🛡️ And it's important that extension writers — that's you, bestie — likewise use the available methods and tools to make sure their extensions don't allow unchecked input to, for example, cause the code to get some kind of runtime exception. not the vibe. 💀

The object system has simplified the task of validating arguments, to some degree, because the object system automatically attempts type conversions as needed, and it'll return an error when a type conversion fails. Slay. ✨

A simple but important check that every C extension should do: verify that it has the right number of arguments. No excuses.

The act of trying to use, say, a string as an integer implicitly performs the type conversion of the string and — if it doesn't work as an integer — returns TCL_ERROR. The developer should check for the TCL_ERROR return from all of the GetXxxFromObj commands and handle them as appropriate. Usually that means propagating the error on back to the user, or to an intervening catch, as the case may be.

You should also check that values are in range (when their ranges are known), and so forth. And when C data structures need to be handled in Tcl in some form or another — yet the contents of the data must remain opaque to Tcl, as is usually the case with binary data (although future releases of Tcl are expected to have native abilities to read, write and manipulate binary data intrinsically) — then handles need to be used. Handles get described, with examples, later in this doc. 🫶

🔥 ANOTHER C EXTENSION — THE MAX COMMAND 🔥

Okay bestie, new command, new vibe. Here's the deal: you hand this one two or more arguments, it compares them all, and the one with the max value gets returned. If everything goes well, that is. ✨

It's an error if you give it fewer than two arguments — and yeah, that counts the pointer to the max command text itself, objv[0], plus a pointer to at least one object whose value we actually compare. Bestie, one arg is not a comparison, it's a monologue. 💅

int
Tcl_MaxCmd (clientData, interp, objc, objv)
    char         *clientData;
    Tcl_Interp   *interp;
    int           objc;
    Tcl_Obj     **objv;
{
    int maxVal = MININT;
    int value, idx;

    if (objc < 3)
     return TclX_WrongArgs (interp, objv[0],
                               " num1 num2 [..numN]");

    for (idx = 1; idx < objc; idx++) {
        if (Tcl_GetIntFromObj (interp, objv[idx], &value) != TCL_OK)
            return TCL_ERROR;

        if (value > maxVal) {
            maxVal = value;
        }
    }
 Tcl_SetIntObj (Tcl_GetObjResult (interp), value);
    return TCL_OK;
}

So this is where we meet the Extended Tcl helper function TclX_WrongArgs. 🎀 It's giving "we need to be called with a wrong number of args" energy — like, this routine makes it so easy to build an error message and an error return in response to that super common mistake of getting called with the wrong arg count. Lowkey iconic that somebody wrapped that up for us.

Then Tcl_GetIntFromObj is doing the fetching — it grabs the integer values of all the remaining arguments. If any of them fail to convert, we return a Tcl error, no hesitation. 🛑 And heads up: if you specify an interpreter in the call to Tcl_GetIntFromObj, an appropriate error message about the conversion failure gets left in the result — so yeah, that's exactly what we do here.

And after we've gone through every single argument looking for the largest value, we set the result object to contain that value, and return TCL_OK. Slay. 💖

🌈 RETURNING RESULTS 🌈

When Tcl-callable functions finish up, they should normally return TCL_OK or TCL_ERROR. Simple as that, bestie.

  • TCL_OK — you return this when the command succeeded. 🫶
  • TCL_ERROR — you return this when the command failed in some abnormal way. 💀

And TCL_ERROR is the move for all syntax errors, for non-numeric values when numeric ones were expected, and so forth. All of it, no exceptions.

Where it gets a lil less clear in some cases is deciding whether Tcl errors should be returned, or whether a function should just hand back a status value. Example, since y'all like examples: end-of-file during a gets returns a status, but open returns an error if it fails. That's just how it is. 🙈

Good news though — errors can be caught from Tcl programs using the catch command. (Go check Tcl's catch(n) and error(n) manual pages.)

The less common return values are TCL_RETURN, TCL_BREAK and TCL_CONTINUE. You use these if you're out here adding new control and/or looping structures to Tcl. To see these values in action, go examine the source code to Extended Tcl's loop commands. Fun fact: Tcl's while, for and if commands used to work in the exact same manner, but they're now compiled into bytecode by the bytecode for performance. ⭐

🦋 ANOTHER C EXTENSION — THE LREVERSE COMMAND 🦋

Next up, a command where you pass a list as an argument and get back a list containing all of the elements of that list in reverse order. Main character reversal energy, honestly.

It's an error if anything other than two arguments are passed — and yes, that's the pointer to the lreverse command text itself, objv[0], plus a pointer to the list to reverse. Two. That's it. That's the number. ✋

Once lreverse has determined that it received the correct number of arguments, Tcl_ListObjGetElements gets called to crack the list open into its own objc count of elements and objv array of pointers to the list's elements.

Then lreverse operates on that array of pointers, swapping them from lowest to highest, second-lowest to second-highest, and so forth. Swapping besties, basically. 🔁

Tcl_ListObjAppendElement gets called on successive list elements to build up the new list, which is finally returned as the result of the command.

int
Tcl_LreverseObjCmd(notUsed, interp, objc, objv)
    ClientData    notUsed;        /* Not used. */
    Tcl_Interp   *interp;         /* Current interpreter. */
    int           objc;           /* Number of arguments. */
    Tcl_Obj     **obj;            /* Argument strings. */
{
    int listObjc, lowListIndex, hiListIndex;
    Tcl_Obj **listObjv;
    char *temp, *resultList;
    Tcl_Obj **newListObjv;

    /* Verify argument count.  Since we take only one argument, argument
     * count must be 2 (command plus one argument).
     */
    if (objc != 2)
     return TclX_WrongArgs (interp, objv [0], "list");

    /* Create an object to handle the new list we're creating */
    newListObjv = Tcl_NewObj();

    /* Crack the list at objv[1] into its own count and array of object
     * pointers.
     */
    if (Tcl_ListObjGetElements (interp, objv[1], &listObjc, &listObjv) != TCL_OK) {
     return TCL_ERROR;
    }

    /* For each element in the source list from last to first, append an
     * element to the new list.
     */
    for (listIndex = listObjc - 1; listIndex >= 0; listIndex--) {
     Tcl_ListObjAppendElement (interp, newListObjv, listObjv[listIndex]);
    }
FIX: NEED TO RETURN THE LIST.
    return TCL_OK;
}
Okay no cap — that big ol' FIX: NEED TO RETURN THE LIST. line is sitting right there in the original, exactly like that. I am not touching it, not fixing it, not editing it. It stays. Copy it character for character if you're copying this code. 😭

💅 INSTALLING YOUR COMMAND 💅

To install your command into Tcl you must call Tcl_CreateObjCommand, and you pass it:

  • a pointer to the interpreter you want to install the command into 🌸
  • the name of the command
  • a pointer to the C function that implements the command
  • a client data pointer
  • a pointer to an optional callback routine

The client data pointer and the callback routine get described later, bestie, don't worry. (That's the next section. Patience is a virtue, iykyk.)

So like, for the max function above — which, incidentally, comes from TclX's tclXmath.c in the TclX7.4/src directory 👑

Tcl_CreateCommand (interp, "max", Tcl_MaxCmd, (ClientData)NULL,
                  (void (*)())NULL);

In the example up there, the max function gets added to the specified interpreter, and both the client data pointer and callback function pointer are NULL. Clean, minimal, no extra baggage. 💖

(For the complete full-on tea on Tcl_CreateCommand and its companion routine Tcl_CommandInfo, please go examine the CrtCommand(3) command page in the core Tcl distribution. That's where the lore lives.)

🧚 DYNAMIC STRINGS 🧚

Dynamic strings are, and I cannot stress this enough, an important abstraction that first became available with Tcl 7.0. 🌟

Dynamic strings — or DStrings, for the girlies in a hurry — provide a way to build up arbitrarily long strings through a repeated process of appending information to them. DStrings reduce the amount of allocating and copying required to add information to a string. And further? They simplify the whole process of doing so. So it's not just fast, it's easy. Ate. 🎀

At first glance it may seem like the object system supersedes DStrings. It does not, and here's why: the performance improvements made possible by the lazy conversion of an object's representation from one datatype to another don't really come into play much while you're constructing strings, because the string representation is always available either without any type conversion, or in the case where type conversion would be necessary anyway, since a string representation of the object is required when strings are being constructed by concatenation, etc.

It should be noted, however — and this is the good part — that the C level string manipulation capabilities of objects, such as Tcl_AppendToObj and Tcl_AppendStringsToObj, are often plenty enough for what you need to do. Like, you might not even need the fancy stuff. ✨

For complete information on dynamic strings, please examine the DString(3) manual page in the core Tcl distribution. And for more on Tcl object's string-oriented calls, go seek Tcl_StringObj(3) in the same location. Reading is fundamental bestie. 📚

👑 CLIENT DATA 👑

The client data pointer provides a means for Tcl commands to have data associated with them that is not global to the C program and not included in the Tcl core. Very demure, very scoped.

Client data is essential in a multi-interpreter environment — that's where a single program has created and is making use of multiple Tcl interpreters — so that C routines can maintain any permanent data they need on a per-interpreter basis. 🫶

And here's the warning, bestie, listen up: if the static data you needed was simply declared static in C, you will probably have reentrancy problems when you work with multiple interpreters. 💀 Not a vibe. Not the move.

Tcl solves this through the client data mechanism. Here's the play-by-play:

When you're about to call Tcl_CreateObjCommand to add a new command to an interpreter, and your command needs to keep some read/write data across invocations, you should allocate that space — preferably using Tcl_Alloc instead of malloc — and then pass the address of that space as the ClientData pointer to Tcl_CreateObjCommand. 🎯

Then, when your command gets called from Tcl, the ClientData pointer you passed to Tcl_CreateObjCommand will be passed to your C routine through the ClientData pointer calling argument. Full circle. Iconic.

And commands that need to share this data with one another can totally do so by using the same ClientData pointer when the commands get added. Sharing is caring, fr fr. 💖

It's super important to note that the Tcl extensions in the tclX8.0.0 directory have had all of their data set up in this way. And since release 6.2, Extended Tcl has supported multiple interpreters within one invocation of Tcl. History, babes. 🌟

✨ THEORY OF HANDLES ✨

Okay bestie, sometimes you need a data element that is just… not a string. Like a pointer to a whole complex C data structure. And the urge to just pass pointers around inside Tcl as strings (converting to and from hex or integer representations, whatever) is SO tempting and SO not it. You mess one up even a tiny bit and boom — core dump. 💀 Not cute. Do not.

So instead we have the concept of handles. A handle is an identifier that a C extension can hand to Tcl, and accept back from Tcl, so the transition between "what your C code knows this thing as" and "what name Tcl knows it by" is as safe and as painless as humanly possible. 🫶

Example, because examples slay: the I/O system built into Tcl uses file handles. When you open a file from Tcl, you get back a handle of the form filen, where n is a file number. Then when you pass that file handle back to puts, gets, seek, flush and so forth, they validate the file handle by checking that the file text is present, and then converting the file number to an integer that they use to look into a data structure of pointers to Tcl open file structures. And those structures contain a Unix file descriptor, flags saying whether or not the file is currently open, whether the file is a file or a pipe, and so forth. Iconic architecture honestly. 👑

Handles were proven so useful that, since TclX release 6.1a, general support has been available to help you create and manipulate them. A lot of these capabilities have migrated into baseline Tcl. So if you've got a similar need, you might like to use the handle routines documented in Handles(3) in Extended Tcl.

The recommendation, straight up: use a unique-to-your-package textual handle coupled with a specific identifier, and let the handle management routines validate it when it's passed back. Why? Because it is so much easier to track down a bug with an implicated handle named something like file4 or bitmap6 than just plain 6. iykyk. 🔍

Also, note that Tcl's object offers another way for complex data structures to exist in parallel with, and underneath, Tcl strings. As of this writing (May 30, 1997) this is fairly new territory, but things are looking good for the prospects of using the Tcl object system in this manner, and for enhancements to the object system that allow even Tcl objects to have methods in a very straightforward and simple way. ✨

🌸 USING COMMANDS TO DO THE SAME THING, AND MORE 🌸

Another handle-like technique — first popularized in the Tk toolkit — gives you handle-like capabilities plus some neat extra stuff. The move is: create a new Tcl command from C that uses ClientData to keep a "handle" on its complex underlying data structure. 🎀

Then, by having that command look at its second argument for what it is to do (its sub-functions), you get these nice methods — several additional sub-commands that don't pollute the global namespace and only work on (and are only available with) the objects (new commands) they are relevant to. Clean. No mess. Periodt.

For example, in Tk, creating a checkbutton (checkbutton .b) creates a new Tcl command (.b) that has methods to configure the button, select, deselect, toggle and flash it. Obsessed with the ergonomics. 💅

A lot of people think this is really the way to go, and I am pretty much leaning that way myself. If you use the incr tcl script-level object system for Tcl, objects that you define in Tcl will be highly compatible — in terms of their command interfaces and configuration management — with objects you create in C using the command-and-ClientData technique described here. I believe Tk has some nice facilities for making this easy for the Tcl programmer. Itcl certainly does.

🧚 TRACKING MEMORY CORRUPTION PROBLEMS 🧚

Occasionally you may write code that scribbles past the end of an allocated piece of memory. 🙈 This will usually result in a core dump or a memory allocation failure sometime later in the program — and often it implicates code that is not actually responsible for the problem. Because you start looking from the point where the error is detected, which is usually where the later routine has failed. So rude of it.

Good news: the memory debugging routines included in Tcl can help find these problems. Developed by Mark and Karl, the memory debugging routines are now part of baseline Tcl, and are to our knowledge the largest piece of TclX to drop into the core without being reengineered first. 😭✨

(Story time, bestie: summer back in '91, John was sitting in his office in the CS building at UC Berkeley trying to find a memory leak somewhere in Tcl, when he was paid a visit by two long-haired-yet-polite programmers bearing gifts in the form of the technology grab-bag known as Extended Tcl. He saw that, using TclX's malloc routines, Tcl could be prompted to print the filename and line number of every single memory allocation that did not have a corresponding free. It was just what the doctor ordered ;-) )

See Memory(TCL) for details. 🍓

💖 INSTALLING YOUR EXTENSIONS INTO TCL 💖

To add your extensions to Tcl, you used to have to statically link them — together with any other extensions — into a single binary executable image. 😮‍💨

Today, although the statically linked executable is still an option, most operating systems, even Microsoft Windows, support shared libraries. And in most cases, Tcl can now make use of those shared libraries, such that your extensions, and most others, can now be built as shared libraries that can be loaded in (using package require) by scripts that need them.

Shared libraries can simplify a Tcl installation, because only one copy of Tcl is required, rather than a hodgepodge of combinations of applications that you might have found at a big Tcl site in the previous era. Glow-up achieved. 💅

⭐ GNU AUTOCONF ⭐

While the build procedure for shared libraries varies from system to system, most Unix and Unix workalike systems will figure out the nuances of the compiler and linker arguments automatically when the configure script is run.

If you are building a package that you plan to make generally available, we strongly recommend that you use GNU autoconf (ftp://prep.ai.mit.edu/pub/gnu) to set up an automatic configure script for it.

Be forewarned: autoconf uses some pretty heavy duty shell and sed script magic to get the job done, and the learning curve can be pretty steep. 🌈 But once done and shaken out, it's rewarding to know that your package can build and run on everything from a notebook to a Cray to a RISC SMP server. 👑

Application-specific startup is accomplished by creating or editing the Tcl_AppInit function. In Tcl_AppInit you should add a call to an application-specific init function which you create. That function should take the address of the interpreter it should install its commands into, and it should install those commands with Tcl_CreateCommand and do any other application-specific startup that is necessary.

The naming convention for application startup routines is App_Init, where App is the name of your application. For example, to add an application named cute one would create a Cute_Init routine that expected a Tcl_Interp pointer as an argument, and add the following code to Tcl_AppInit:

if (Cute_Init (interp) == TCL_ERROR) {
 return TCL_ERROR;
}

As you can guess from the above example, if your init routine is unable to initialize, it should use Tcl_AppendResult to provide some kind of useful error message back to TclX, then return TCL_ERROR to indicate that an error occurred. If the routine executed successfully, it should return TCL_OK. Simple as. 🫶

When you examine Tcl_AppInit, note that there is one call already there to install an application — the call to TclX_Init installs Extended Tcl into the Tcl core.

🦋 MAKING APPLICATION INFORMATION VISIBLE FROM EXTENDED TCL 🦋

TclX's infox command can return several pieces of information relevant to Extended Tcl, including the application's name, descriptive name, patch level and version. Your application's startup can set these variables to application-specific values. If it doesn't, they are given default values for Extended Tcl.

To set these values, first be sure that you include either tclExtend.h or tclExtdInt.h from the source file that defines your init routine. This will create external declarations for the variables. Then, set the variables in your init routine, for example:

tclAppName = "cute";
tclAppLongName = "Call Unix/Tcl Environment";
tclAppVersion = "2.1";
Note that the default values are set by TclX_Init, so if you wish to override them, you must call your init routine in Tcl_AppInit after its call to TclX_Init. Order matters, bestie. 🔥

🌈 EXTENDED TCL EXIT 🌈

When Extended Tcl exits, Tcl_DeleteInterp may be called to free memory used by Tcl. Normally this is only called if TCL_MEM_DEBUG was defined, since Unix will return all of the allocated memory back to the system anyway. If TCL_MEM_DEBUG was defined, it is called so that any memory that was allocated without ever being freed can be detected. This greatly reduces the amount of work to detect and track down memory leaks — that situation where some piece of your code allocates memory repeatedly without ever freeing it, or at least without always freeing it. 💀

It is often necessary for an application to perform special cleanup functions upon the deletion of an interpreter as well. To facilitate this activity, Tcl provides the ability to perform a function callback when an interpreter is deleted. To arrange for a C function to be called when the interpreter is deleted, call Tcl_CallWhenDeleted from your application initialization routine. For details on how to use this function, read the CallDel(3) manual page that ships with core Tcl. 📖

✨ EXECUTING TCL CODE FROM YOUR C EXTENSION ✨

okay bestie, picture this: you're deep in your C extension, vibing, and then you realize you need some operation performed — one that would be so easy from Tcl but possibly excruciating to do directly in C. 😭 Tcl gets it. It provides a whole bunch of C-level interfaces so you can just... make Tcl code execute. 👑

The old-style calls are Tcl_Eval, Tcl_VarEval, Tcl_EvalFile and Tcl_GlobalEval. And the results of those calls? You can dig them right out of the interpreter using Tcl_GetStringResult if you want a string representation of the result, or Tcl_GetObjResult if you want the object itself. 👀 Also btw — using interp->result to get at the result string is deprecated, so like... don't do that. 🙈

Then the Tcl object system rolls up and adds Tcl_EvalObj and Tcl_GlobalEvalObj. 💖 The difference? You're evaluating an object, not just a string, and choosing these over the ones we just mentioned can give you a major performance improvement in your code when that code gets executed repeatedly — and yes, that includes when it only executes once but loops several times within itself 🌀 — because these routines make it possible for the bytecode compiler to compile the code being evaluated and save that compiled code along with the data structure, in an implementation-dependent manner. no cap.

Want the full lore? Please consult the EvalObj(3) and Eval(3) manual pages within the Tcl distribution. 📚

🌸 ACCESSING TCL VARIABLES AND ARRAYS FROM YOUR C EXTENSIONS 🌸

So besides the non-object-system ways of reading from and storing to Tcl variables — you know, using routines like Tcl_SetVar2 and Tcl_GetVar2 — Tcl variables and arrays can also be read from a C extension as Tcl objects using the Tcl_ObjGetVar2 function, and set from C extensions through the Tcl_ObjSetVar2 function. 🫶

but bestie, heads up: the object versions do not carry forward analogues to the one-variable-name-argument Tcl_GetVar, Tcl_SetVar, and Tcl_UnsetVar. 😬 So if you know you have a scalar, just call the object variable get and set functions with a NULL second argument. And if your variable name might contain an array reference via a self-contained embedded array index — like, I'm asking Tcl_ObjGetVar2 for "foo(5)" instead of "foo" "5" — then add the TCL_PARSE_PART1 to the flags in your call. 💅

Now, the fact that Tcl_ObjGetVar2 retrieves Tcl objects instead of strings is critical for the object system being able to provide those performance boosts from "lazy" type conversion and the binary data capabilities. ✨ But the arguments containing the variable name — or the array name and element name if they've been split out — also have to be specified as Tcl objects rather than strings. While that's useful on occasion, those of us writing C extensions for Tcl in the post-object-system era usually have the names available as plain old char * variables, which means converting the strings to objects before use and accounting for their possible destruction afterwards. 😮‍💨

To simplify the task in those cases, TclX adds the TclX_ObjGetVar2S subroutine. It works just like Tcl_ObjGetVar2, except the one or two variable name arguments are specified as strings, and the routine takes care of making and disposing of object equivalents for you. iconic behavior, honestly. 👑

Tcl variables can be unset from C via the Tcl_UnsetVar and Tcl_UnsetVar2 functions. There are currently (as of 8.0) no object-system equivalents, so in the rare case where you have the name of the variable you want unset as an object instead of a string, you can call Tcl_GetStringFromObj to obtain the string representation first. 🙃

For complete information on these functions, please refer to the ObjSetVar(3) and SetVar(3) manual pages in the doc directory of the core Tcl distribution. 💜

🎀 LINKING TCL VARIABLES TO C VARIABLES 🎀

Tcl_LinkVar and Tcl_UnlinkVar can be used to automatically keep Tcl variables synchronized with corresponding C variables. 🔗 Once a Tcl variable has been linked to a C variable with Tcl_LinkVar, anytime the Tcl variable is read, the value of the C variable is converted (if necessary) and returned, and when the Tcl variable is written, the C variable will be updated with the new value. it's giving ✨effortless two-way sync✨.

Tcl_LinkVar uses variable traces to keep the Tcl variable named by varName in sync with the C variable at the address given by addr.

int, double, boolean and char * variables are supported. 🌈 You can make your linked variables read only from the Tcl side, as well. And ngl, this next part is serious: the C variables must continually exist while they are linked. In other words, linking "automatic" C variables — the ones created on the stack while a routine is being executed and destroyed afterwards — will result in a malfunctioning program at best and a coredump or more at worst. 💀 do not do this. fr fr.

For more information, please examine the LinkVar(3) manual page in the core Tcl distribution. 📖

🧚 ADDING NEW MATH FUNCTIONS TO TCL 🧚

As of Tcl version 7.0, math functions such as sin, cos, etc, are directly supported within Tcl expressions. ✨ These obsolete the Extended Tcl commands that provided explicit commands for these functions for many, many releases — although procs equivalencing the old TclX commands to the new math functions are still provided for backwards compatibility. 🫶 we love a graceful era transition.

New math functions can be added to Tcl, or existing math functions can be replaced, by calling Tcl_CreateMathFunc. 💅

🦋 ACCESSING AND MANIPULATING THE RANDOM NUMBER GENERATOR 🦋

Prior to Tcl version 8.0, the Tcl core did not provide access to a random number generator, but TclX did, through its random command. 🎲 As of Tcl version 8.0, access to a random number generator is provided by baseline Tcl through the new math functions, rand and srand. periodt.

The TclX random command is still available — it has some useful capabilities not directly provided by the new baseline functions. 👑

For more information on adding your own math functions to Tcl, please study the CrtMathFnc(3) manual page in the core Tcl distribution. 📚

🌈 CONVERTING FILENAMES TO NATIVE FORM AND PERFORMING TILDE SUBSTITUTIONS 🌈

The Tcl_TranslateFileName function is available to C extension writers to translate filenames to a form suitable for use by the local operating system. 🗂️ It converts network names to their native form, and if the name starts with a ~ character, the function returns a new string where the name is replaced with the home directory of the given user. 🏠✨

For more information please consult the Translate(3) manual page in the core Tcl distribution.

💜 SETTING THE RECURSION LIMIT 💜

Tcl has a preset recursion limit that limits the maximum allowable nesting depth of calls within an interpreter. 🌀 This is useful for detecting infinite recursions before other limits — like the process memory limit or, worse, available swap space on the system — run out. 😭 genuinely a protective queen.

The default limit is just a guess, however, and applications that make heavy use of recursion may need to call Tcl_SetRecursionLimit to raise this limit. 📈

For more information, please consult the SetRecLmt(3) manual page in the core Tcl distribution.

💅 HANDLING SIGNALS FROM TCL EXTENSIONS 💅

So if an event like a signal occurs while a Tcl script is being executed, it isn't safe to do much in the signal handling routine — the Tcl environment cannot be safely manipulated at this point because it could be in the middle of some operation, such as updating pointers, leaving the interpreter in an unreliable state. 😬 scary but true.

The only safe approach is to set a flag indicating that the event occurred, then handle the event later when the interpreter has returned to a safe state, such as after the current Tcl command completes. 🚩➡️✅

The Tcl_AsyncCreate, Tcl_AsyncMark, Tcl_AsyncInvoke, and Tcl_AsyncDelete functions provide a safe mechanism for dealing with signals and other asynchronous events. For more information on how to use this capability, please refer to the Async(3) manual page in the core Tcl distribution. 📖

Note that Extended Tcl provides built-in support for managing signals in numerous ways, including generating them with alarm(2) and kill(2), ignoring them, trapping them, getting, setting, blocking and unblocking them. 🤯 You can cause specific code to execute at a safe point after a signal occurs, or cause a Tcl error backtrace on one's occurrence. For more information, please examine the TclX documentation. 👑

🍓 PARSING BACKSLASH SEQUENCES 🍓

The Tcl_Backslash function is called to parse Tcl backslash sequences. 🌙 These backslash sequences are the usual sort that you see in the C programming language — such as \n for newline, \r for return, and so forth. Tcl_Backslash parses a single backslash sequence and returns a single character corresponding to the backslash sequence. so satisfying, honestly.

For more info on this call, look at the Backslash(3) manual page in the core Tcl distribution. And for information on the valid backslash sequences, consult the summary of Tcl language syntax, Tcl(n) in the same distribution. 📚

⭐ HASH TABLES ⭐

Hash tables provide Tcl with a high-performance facility for looking up and managing key-value pairs located and maintained in memory. 🗝️ Tcl uses hash tables internally to locate procedure definitions, Tcl variables, array elements, file handles and so forth. And Tcl makes the hash table functions accessible to C extension writers as well. 🫶 sharing is caring.

Hash tables grow automatically to maintain efficiency, rather than exposing the table size to the programmer at allocation time — which would needlessly add complexity to Tcl and would be prone to inefficiency due to the need to guess the number of items that will go into the table, and the seemingly inevitable growth in amount of data processed per run over the useful life of the program. 📈 it's giving ✨self-managing✨.

For more information on hash tables, please consult the Hash(3) manual page in the core Tcl distribution.

🔥 TRACING VARIABLE ACCESSES 🔥

The C extension writer can arrange to have a C routine called whenever a Tcl variable is read, written, or unset. 👀 Variable traces are the mechanism by which Tk toolkit widgets such as radio and checkbuttons, messages and so forth update without Tcl programmer intervention when their data variables are changed. They are also used by the routine that links Tcl and C variables, Tcl_LinkVar, described above. 🔗

Tcl_TraceVar is called to establish a variable trace. Entire arrays and individual array elements can be traced as well. 🎯 If the programmer already has an array name in one string and a variable name in another, Tcl_TraceVar2 can be called. Calls are also available to request information about traces and to delete them. 🗑️

For more information on variable traces, consult the TraceVar(3) manual page in the core Tcl distribution. 💖

🌀 TRACING TCL EXECUTION 🌀

okay so bestie, Tcl can literally call C routines every single time it runs a Tcl command — up to a depth of nesting levels you specify. it's giving surveillance but make it cute 💅

  • Tcl_CreateTrace — makes an execution trace
  • Tcl_DeleteTrace — deletes it

Command tracing is what Extended Tcl uses under the hood to implement the cmdtrace Tcl command, which is a genuinely useful lil guy for debugging your Tcl applications.

For the full tea on execution tracing, go look at the CrtTrace(3) manual pages in the core Tcl distribution. 📖

🧮 EVALUATING TCL EXPRESSIONS FROM C 🧮

Tcl_ExprLong, Tcl_ExprDouble, Tcl_ExprBool, and Tcl_ExprString all take string arguments and, when you call them, evaluate those strings as Tcl expressions. Depending on which routine you called, the result is either a C long, a double, a boolean (an int with a value of 0 or 1), or a char * (which you get through Tcl_GetResult). ✨

Now — to take advantage of the performance gains available through the bytecode compiler, Tcl_ExprLongObj, Tcl_ExprDoubleObj, Tcl_ExprBoolObj, and Tcl_ExprObj all take an object containing an expression to be evaluated (rather than a string). The result? The bytecode-compiled version of the expression gets kept in the object, right alongside the string representation. If the expression gets evaluated again without being changed, it does not have to be recompiled... a major performance win, no cap. 🔥

For complete info on evaluating Tcl expressions from C, you're invited to examine the ExprLong(3) and ExprLongObj(3) manpages in the core Tcl distribution.

🎯 PATTERN MATCHING 🎯

The Tcl_StringMatch function can be called to see if a string matches a specified pattern. Tcl_StringMatch is called by the Tcl string match command, so the pattern format is identical. The pattern format is similar to the one used by the C-shell; string(n) describes this format.

More info about Tcl_StringMatch is available in the StrMatch(3) manpage in the core Tcl distribution.

🔍 REGULAR EXPRESSION PATTERN MATCHING 🔍

Tcl_RegExpMatch can be called to determine whether a string matches a regular expression. Tcl_RegExpMatch is used internally by the regexp Tcl command.

Since regular expressions are typically "compiled" before use — a fairly involved process — Tcl also supports routines that separate the compilation of an expression from its use: Tcl_RegExpCompile, Tcl_RegExpExec, and Tcl_RegExpRange. If an expression is going to be matched many times, doing the compile once and caching the compiled regular expression result, then reusing the cached version by using Tcl_RegExpExec, can be a significant performance win. 💫

For more info on this function, please consult the RegExp(3) manpage in the core Tcl distribution.

📋 MANIPULATING TCL LISTS FROM C EXTENSIONS 📋

The C extension writer often needs to create, manipulate and decompose Tcl lists. Tcl_SplitList and Tcl_Merge used to be the only way to parse strings into lists and vice versa. As of Tcl 8, lists can be parsed and assembled, object-style, using Tcl_ListObjGetElements and Tcl_SetListObj, and friends. 🫶

Once again, the "win" of using object-system-based list manipulation instead of the previous string-based routines is that the parsing of a string in an object to a list gets cached in the object structure — same as with integers and floating point numbers, compiled procedures, etc. The next time this string needs to be looked at as a list, if the contents of the string haven't changed, the string doesn't have to be parsed. periodt.

In the author's experience, working with an admittedly degenerate test whereby they iterated rather inefficiently across a 6,000-element list, a speedup factor of more than 2500 was obtained over the previous non-object-based version of Tcl. 😭 like... she ATE.

For more info on these commands, please consult the ListObj(3) manual page in the core Tcl distribution.

🧵 CONCATENATING STRINGS 🧵

Tcl_ConcatObj concatenates the string representation of zero or more objects into a single new object. The elements of the new string are space-separated. Tcl_Concat does the same thing for strings, as Tcl_ConcatObj does for objects.

Concatenating strings is similar to constructing lists from them, except that Tcl_ConcatObj and Tcl_Concat do not attempt to make the resulting string into a valid Tcl list. 💅

Tcl_Concat is documented in the Concat(3) manpage, and Tcl_ConcatObj in the tringObj manpage, both in the core Tcl distribution.

✅ DETECTING WHETHER OR NOT YOU HAVE A COMPLETE COMMAND ✅

C routines that collect data to form a command to be passed to Tcl_Eval often need a way to tell whether they have a complete command already or whether they need more data. (Programs that read typed-in Tcl input such as Tcl shells need this capability, for instance.) Tcl_CommandComplete can be used to tell whether or not you have a complete command. vibe check: complete or not complete, that is the question.

For more info examine CmdCmplt(3) in the core Tcl distribution.

📜 RECORDING COMMANDS FOR COMMAND HISTORY 📜

Tcl has a history mechanism that's accessed from Tcl through the history command. If you want your extension to propagate commands into the command history, you should call Tcl_RecordAndEvalObj (object system) or Tcl_RecordAndEval (old system),

These commands work like Tcl_EvalObj and Tcl_Eval, respectively, except that these versions record the command as well as executing it. 🎀

Tcl_RecordAndEval and Tcl_RecordAndEvlObj should only be called with user-entered top-level commands, since the history mechanism exists to allow the user to easily access, edit and reissue previously issued commands. 👑

bestie please don't call these on nested/internal stuff, that's literally not what the history mechanism is for, it's giving chaos

For complete info on these functions, please examine the RecordEval.3 and RecEvalObj.3 manual pages in the core Tcl distribution.

🔢 CONVERTING FLOATING POINT VALUES TO STRINGS 🔢

The Tcl object system's Tcl_GetDoubleFromObj and Tcl_SetDoubleObj use Tcl objects, rather than the strings used by Tcl_PrintDouble, and convert, when necessary, an ASCII string to a double and back again.

These routines ensure that the string output will continue to be interpretable as a floating point number, rather than an integer, by always putting a `.` or `e` into the string representing the number. 😌 iconic behaviour honestly.

The precision of the output string is controlled by the Tcl tcl_precision variable.

For complete info on these routines, please examine DoubleObj(3) and PrintDbl(3) in the core Tcl distribution.

👶 CREATING CHILD PROCESSES AND PIPELINES FROM C 👶

Tcl_OpenCommandChannel provides a C-level interface to the exec and open commands. The child (or pipeline of children) can have its standard input, output and error redirected from files, variables or pipes. To understand the meaning of the redirection symbols understood by this function, look at the exec(n) Tcl command. For complete info on Tcl_OpenCommandChannel, please examine OpenFileChnl(3).

📂 ACCESSING TCL FILEHANDLES FROM C 📂

On Posix/Unix systems, Tcl filehandles passed to your C extension can be translated to a Posix FILE * structure using the Tcl_GetOpenFile function, documented in GetOpnFl.3.

🧹 MANAGING BACKGROUND PROCESS TERMINATION AND CLEANUP 🧹

When a Posix system does a fork to create a new process, the process ID of the child is returned to the caller. After the child process exits, its process table entry (and some other data associated with the process) cannot be reclaimed by the operating system until a call to waitpid, or one of a couple of other, similar system calls, has been made by the parent process. 💀 zombie processes are literally the undead and we do not stan.

The C extension writer who has created a subprocess, by whatever mechanism, can turn over responsibility for detecting the processes' termination and calling waitpid to obtain its exit status, by calling Tcl_DetachPids on it.

Tcl_ReapDetachedProcs is the C routine that will detect the termination of any processes turned over to Tcl, permitting the processes to be fully reclaimed by the operating system. It is usually not necessary to call Tcl_ReapDetachedProcs, as it is called automatically every time exec is performed. ✨

For complete info on these routines, please look at DetachPids(3) in the core Tcl distribution.

📚 FOR MORE INFORMATION 📚

In addition to the documentation referenced above, you can learn a lot by studying the source code of the commands added by Tcl, Tk and Extended Tcl, etc. 📖

The comp.lang.tcl Usenet newsgroup is read by hundreds of thousands of Tcl people. A number of Frequently Asked Questions (FAQs) about Tcl are posted there periodically. The newsgroup is a good place to ask questions (after you've made sure they're not already answered in the FAQ ;-) 🫶

Finally, if you have interactive Internet access, you can ftp to ftp://ftp.neosoft.com/pub/tcl, the site for contributed Tcl sources. This site contains quite a few extensions, applications, and so forth, including several object-oriented extension packages. 🌈

If you have access via the world-wide web, check out the Sun Microsystems site (http://sunscript.sun.com), the contributed sources archive website (http://www.neosoft.com/tcl), and the homepage for Extended Tcl (http://www.neosoft.com/tclx). 🌸

👑 AUTHORS 👑

Extended Tcl was created by Karl Lehenbauer (karl@neosoft.com) and Mark Diekhans (markd@grizzly.com).

and that's a wrap on this whole entire saga bestie!! 🎉 this page lives in the Tcl documentation set, filed under Command Writing(3tcl) — go off and write some absolutely unhinged C extensions for Tcl, you're literally the main character now. mwah 💋 🦋 ⭐

📦 where this page comes from