Tcl_Obj(3tcl)

📦
version 9.0.4 · 3 versions
, in ubuntu stonking

🌸 Tcl_Obj 🌸 — the Tcl Library Procedures era, bestie

ok so like. this is the manual page for all the Tcl_Obj-adjacent core functions in Tcl (manual section 3tcl), and honestly? it's a lot. gird your loins, grab a strawberry 🍓, and let's go.

💖 NAME 💖

Tcl_NewObj, Tcl_DuplicateObj, Tcl_IncrRefCount, Tcl_DecrRefCount, Tcl_BounceRefCount, Tcl_IsShared, Tcl_InvalidateStringRep — all of them literally just manipulate Tcl values. that's the whole job description. periodt.

✨ SYNOPSIS ✨

#include <tcl.h>

Tcl_Obj *
Tcl_NewObj()

Tcl_Obj *
Tcl_DuplicateObj(objPtr)

Tcl_IncrRefCount(objPtr)

Tcl_DecrRefCount(objPtr)

Tcl_BounceRefCount(objPtr)

int
Tcl_IsShared(objPtr)

Tcl_InvalidateStringRep(objPtr)

🎀 ARGUMENTS 🎀

  • Tcl_Obj *objPtr (in) — points to a value. and listen, it must have been the result of a previous call to Tcl_NewObj. like, it's not optional, bestie, that's the whole rule.

🦋 INTRODUCTION 🦋

N.B. Refer to the Tcl_UniChar documentation page for a description of the TUTF-8 encoding and related terms referenced here. (iykyk)

okay so this page is basically the ~lore drop~ on Tcl values — which we call Tcl_Objs, for historical reasons 💀 — and how they get used in the wild. it also walks you through the generic procedures for managing those values: creating them, copying them, and bumping the count of references (aka pointers) to them up and down. these play nice with the procedures that operate on specific types of values, like Tcl_GetIntFromObj and Tcl_ListObjAppendElement. every individual procedure gets described right alongside the data structures it messes with. iconic behavior, honestly.

Tcl's dual-ported values are a general-purpose mechanism for storing and exchanging Tcl values. they've largely replaced the whole "just use strings for everything" situation. like, they're what holds variable values, command arguments, command results, and scripts. Tcl values behave like strings but they ALSO hold an internal representation that can be manipulated more efficiently. for example, a Tcl list is now represented as a value that holds the list's string representation plus an array of pointers to the values for each list element. dual-ported values dodge most runtime type conversions, and they speed up a bunch of operations since an appropriate representation is immediately available. the compiler itself uses Tcl values to cache the instruction bytecodes that come out of compiling scripts. no cap. 🔥

the two representations are a cache of each other, and they're computed lazily. meaning: each representation is only computed when it's actually necessary, it's computed from the other representation, and once it's computed it gets saved. also, a change in one representation invalidates the other one. example time: a Tcl program doing integer calculations can operate directly on a variable's internal machine integer representation without constantly converting between integers and strings. only when it genuinely needs a string representing the variable's value — say, to print it — will the program regenerate the string representation from the integer. so even though values contain an internal representation, their semantics are defined in terms of strings: an up-to-date string can always be obtained, and any change to the value will show up in that string when the value's string representation is fetched. and because of all this invalidation-and-regeneration business, it is dangerous for extension writers to access Tcl_Obj fields directly. don't do it. access Tcl_Obj info using procedures like Tcl_GetStringFromObj and Tcl_GetString instead. 🫶

values are allocated on the heap and referenced using a pointer to their Tcl_Obj structure. values get shared as much as possible, which significantly reduces storage requirements because some values — like long lists — are VERY large, and most Tcl values are only read and never modified. this is especially true for procedure arguments, which can be shared between the caller and the called procedure. assignment and argument binding is done by simply assigning a pointer to the value. reference counting is what determines when it's safe to reclaim a value's storage. 🧚

and yes, Tcl values are typed. a value's internal representation is controlled by its type. several types are predefined in the Tcl core, including integer, double, list, and bytecode. extension writers can extend the set of types by defining their own Tcl_ObjType structs. go off, king. 👑

👑 THE TCL_OBJ STRUCTURE 👑

every Tcl value is represented by a Tcl_Obj structure, which is defined as follows:

typedef struct {
    Tcl_Size refCount;
    char *bytes;
    Tcl_Size length;
    const Tcl_ObjType *typePtr;
    union {
        long longValue;
        double doubleValue;
        void *otherValuePtr;
        Tcl_WideInt wideValue;
        struct {
            void *ptr1;
            void *ptr2;
        } twoPtrValue;
        struct {
            void *ptr;
            unsigned long value;
        } ptrAndLongRep;
        struct {
            void *ptr;
            Tcl_Size size;
        } ptrAndSize;
    } internalRep;
} Tcl_Obj;

the bytes and the length members together hold a value's TUTF-8 byte sequence representation. bytes points to the first byte of the string representation, and length gives the number of bytes. the byte array must always have a null byte after the last data byte, at offset length — that's what lets string representations be treated as conventional null-terminated C strings. C programs use Tcl_GetStringFromObj and Tcl_GetString to get a value's string representation. if bytes is NULL, the string representation is invalid. 💀

a value's type manages its internal representation. the typePtr member points to the Tcl_ObjType structure that describes the type. if typePtr is NULL, the internal representation is invalid.

the internalRep union member holds a value's internal representation. this is either a (long) integer, a double-precision floating-point number, a pointer to a value containing additional information needed by the value's type to represent the value, a Tcl_WideInt integer, two arbitrary pointers, a pair made up of a pointer and an unsigned long integer, or a pair made up of a pointer and Tcl_Size — which is a signed integer type capable of holding the maximum lengths permitted in Tcl. so many options, so little time. ✨

the refCount member is what tells you when it's safe to free a value's storage. it holds the count of active references to the value. maintaining the correct reference count is a key responsibility of extension writers, bestie. reference counting gets its own whole discussion below, in the section STORAGE MANAGEMENT OF VALUES.

even though extension writers can directly access the members of a Tcl_Obj structure, it's much better to use the appropriate procedures and macros. like, extension writers should never read or update refCount directly; use macros such as Tcl_IncrRefCount and Tcl_IsShared instead. 💅

a key property of Tcl values is that they hold two representations. a value typically starts out containing only a string representation: it's untyped and has a NULL typePtr. a value containing an empty string, or a copy of a specified string, is created using Tcl_NewObj or Tcl_NewStringObj respectively. a value's string value is gotten with Tcl_GetStringFromObj or Tcl_GetString, and changed with Tcl_SetStringObj. if the value later gets passed to a procedure like Tcl_GetIntFromObj that requires a specific internal representation, that procedure will create one and set the value's typePtr. the internal representation is computed from the string representation. a value's two representations are duals of each other: changes made to one are reflected in the other. for example, Tcl_ListObjReplace will modify a value's internal representation, and the next call to Tcl_GetStringFromObj or Tcl_GetString will reflect that change. 🪞

representations are recomputed lazily for efficiency. a change to one representation, made by a procedure such as Tcl_ListObjReplace, is not reflected immediately in the other representation. instead, the other representation is marked invalid so that it's only regenerated if it's needed later. most C programmers never have to care about how that's done and simply use procedures such as Tcl_GetBooleanFromObj or Tcl_ListObjIndex. but programmers who implement their own value types must check for invalid representations and mark representations invalid when necessary. the procedure Tcl_InvalidateStringRep is used to mark a value's string representation invalid and to free any storage associated with the old string representation. bye bye, old bytes 🫡

values usually stay one type over their whole life, but occasionally a value has to convert from one type to another. for example, a C program might build up a string in a value with repeated calls to Tcl_AppendToObj, and then call Tcl_ListObjIndex to extract a list element from the value. the same value holding the same string value can have several different internal representations at different times. extension writers can also force a value to convert from one type to another using the Tcl_ConvertToType procedure. only programmers who create new value types need to be concerned with how this is done: a procedure defined as part of the value type's implementation creates a new internal representation for a value and changes its typePtr. see the man page for Tcl_RegisterObjType to see how to create a new value type. 🌈

🎀 EXAMPLE OF THE LIFETIME OF A VALUE 🎀

as an example of the lifetime of a value, consider the following sequence of commands:

set x 123

this assigns to x an untyped value whose bytes member points to 123 and whose length member contains 3. the value's typePtr member is NULL.

puts "x is $x"

x's string representation is valid (since bytes is non-NULL) and gets fetched for the command.

incr x

the incr command first gets an integer from x's value by calling Tcl_GetIntFromObj. this procedure checks whether the value is already an integer value. since it's not, it converts the value by setting the value's internal representation to the integer 123 and setting the value's typePtr to point to the integer Tcl_ObjType structure. both representations are now valid. incr increments the value's integer internal representation, then invalidates its string representation (by calling Tcl_InvalidateStringRep) since the string representation no longer corresponds to the internal representation. 💥

puts "x is now $x"

the string representation of x's value is needed and gets recomputed. the string representation is now 124 and both representations are again valid. slay. ✨

🧺 STORAGE MANAGEMENT OF VALUES 🧺

okay bestie, this is the part where it gets ✨real✨. Tcl values live on the heap, and Tcl is obsessed with sharing them as much as possible so it doesn't eat up all your storage. To keep track of who's still holding onto what, it uses reference counting — that's how it knows when a value is no longer needed and can safely be freed. very demure, very mindful.

A value that was just created by Tcl_NewObj, Tcl_NewStringObj, or literally any Abstract List command or function, has refCount 0, meaning the object can often just be handed straight to a function like Tcl_SetObjResult, Tcl_ListObjAppendElement, or Tcl_DictObjPut (as a value) with zero explicit reference management — and those are all super common use cases, so slay. (lowkey important: the latter two need the target list or dictionary to be well-formed, but that's usually easy to arrange when the value is being initially constructed, so don't stress.)

Here's the holy trinity of refcount macros:

  • The macro Tcl_IncrRefCount bumps the reference count up whenever a brand new reference to the value gets created.
  • The macro Tcl_DecrRefCount drops the count when a reference is no longer needed. And if the value's reference count falls all the way down to zero, it frees its storage. bye bestie 💀
  • The macro Tcl_BounceRefCount will check if the value has no references at all (i.e. it's in a "new" state) and free the value.

If a value is shared between different code or data structures, its refCount is greater than 1. Incrementing a value's reference count guarantees it won't get freed too early or have its value change by accident. protect her at all costs ✨

🌈 a lil example, because visuals help 🌈

The bytecode interpreter shares argument values between the calling and the called Tcl procedures so it doesn't have to copy values everywhere. It assigns the call's argument values to the procedure's formal parameter variables. While doing that, it calls Tcl_IncrRefCount to increment the reference count of each argument, because there's now a whole new reference to it from the formal parameter. Then when the called procedure returns, the interpreter calls Tcl_DecrRefCount to decrement each argument's reference count. And when a value's reference count drops to less than or equal to zero, Tcl_DecrRefCount reclaims its storage. it's giving housekeeping. 🧹

Most command procedures have never had to care about reference counting, because they use a value immediately and don't hang onto a pointer to it after they return. However — and this is the plot twist — some procedures may return a new value with a refCount of 0. In that situation it is the caller's responsibility to free the value before the procedure returns. Two ways to handle this:

  1. Always call Tcl_IncrRefCount before using the value, then call Tcl_DecrRefCount before returning.
  2. Use Tcl_BounceRefCount after the value is no longer needed or referenced. This macro will free the value if there are no other references to it.

And if you're retaining a pointer to a value inside a data structure, the procedure must be careful to increment its reference count, because that retained pointer is a new reference. no cap.

Examples of procedures that return new values are Tcl_NewIntObj, and commands like lseq, which creates an Abstract List — and an lindex on this list may return a new Obj with a refCount of 0. 👀

Command procedures that directly modify values (think the ones for lappend and linsert) must be careful to copy a shared value before changing it. First they have to check whether the value is shared, by calling Tcl_IsShared. If it is shared, they must copy the value using Tcl_DuplicateObj — which returns a new duplicate of the original value with refCount 0. If the value is not shared, the command procedure "owns" the value and can safely modify it directly. periods. 💅

For example, this code appears in the command procedure that implements linsert. This procedure modifies the list value passed to it in objv[1] by inserting objc-3 new elements before index:

listPtr = objv[1];
if (Tcl_IsShared(listPtr)) {
    listPtr = Tcl_DuplicateObj(listPtr);
}
result = Tcl_ListObjReplace(interp, listPtr, index, 0,
        (objc-3), &(objv[3]));

As another example, incr's command procedure must check whether the variable's value is shared before incrementing the integer in its internal representation. If it is shared, it needs to duplicate the value so it doesn't accidentally change values sitting in other data structures. imagine modifying someone else's variable by accident 😭

And in cases where a value is obtained, used, and then not retained, the value can be freed using Tcl_BounceRefCount. This is functionally equivalent to calling Tcl_IncrRefCount followed by Tcl_DecrRefCount. cute lil shortcut, honestly. 🫶

👀 SEE ALSO 👀

🔑 KEYWORDS 🔑

  • internal representation
  • value
  • value creation
  • value type
  • reference counting
  • string representation
  • type conversion

and that's the whole page bestie!! 💖 this one's proudly serving Tcl 8.5, filed under Tcl_Obj(3tcl), and yeah — you read the entire manual page, which is iconic behavior honestly. go forth and count your references responsibly. slay 🦋✨

📦 where this page comes from