tie(3tcl)

📦
version 2.0 · 3 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 tie — Array persistence 💖

okay so bestie, this is tie, from section 3tcl of the Tcl manual — filed under Tcl Data Structures — and it is genuinely one of the cutest little frameworks you're gonna meet. Let's get into it. 🦋

✨ NAME ✨

tie — Array persistence. that's it, that's the whole vibe. 💅

🌸 SYNOPSIS 🌸

package require Tcl 8.5 9

package require tie ?1.3?

::tie::tie arrayvarname options... dstype dsname...

::tie::untie arrayvarname ?token?

::tie::info ties arrayvarname

::tie::info types

::tie::info type dstype

::tie::register dsclasscmd as dstype

dsclasscmd objname ?dsname...?

ds destroy

ds names

ds size

ds get

ds set dict

ds unset ?pattern?

ds setv index value

ds unsetv index

ds getv index

no cap, that's a lot of little commands but they all pull their weight, promise. 🫶

👑 DESCRIPTION 👑

The tie package gives you a whole framework for making persistent Tcl array variables. and like, quick note before you spiral: the mechanism it hands you is generic enough that you can also use it to spread the contents of Tcl arrays across multiple threads and processes — i.e. communication. so it's doing double duty and it's serving in both. ✨

So yeah, persistence and communication — both of those things get done by tying a Tcl array variable to a data source. examples of data sources? other Tcl arrays, and files. simple as that. 🌸

And here's the part that's lowkey the most iconic thing about it: a single Tcl array variable can be tied to more than one data source. 🎀 that one feature is literally what makes the communication use case possible. the whole plot twist. so how it goes: you just tie several Tcl arrays in many client processes to a Tcl array in a server, and all changes to any of them get distributed to all of them. periodt. 💅

and like, less centralized variants of this are of course possible as well — nobody's forcing the server/client arrangement on you. 💜

🌈 USING TIES 🌈

✨ TIE API ✨

okay bestie, this is the part you actually came here for 💖 This is the basic API you use to build ties between your Tcl array variables and whatever data source you're obsessed with. If you're a casual user, this is the only interface you ever have to care about. Everything after this about internal interfaces? You can scroll right past it, no cap, it's giving "for the devs only" 💅

::tie::tie arrayvarname options... dstype dsname...

This command sets up a tie between the Tcl array whose name is given by arrayvarname and the data source identified by dstype plus its little squad of dsname arguments. Every single change you make to that Tcl array after this command returns gets saved to the data source for safekeeping (or for distribution, whichever vibe you're on) ✨

The result of the command is always a token that identifies the new tie — literally always, bestie. Hold onto it, because you'll use it later to destroy that specific tie 👑

Here's the arg breakdown, no skips:

  • varname arrayvarname (in) — the name of the Tcl array variable you're connecting the new tie to 🎀
  • name|command dstype (in) — this says what type of data source we're accessing. It can be one of log, array, remotearray, file, growfile, or dsource — and yeah, programmers can register extra data source types too, the more the merrier 🌸 Each dstype gets followed by one or more arguments identifying the data source your array is getting tied to.
  • string dsname (in) — the series of dsname arguments that come after the dstype. This is what points at the actual data source we're connecting to, and it has to be appropriate for the chosen type. No mismatched vibes allowed 🙈

The command also understands a bunch of extra options that guide how the connection between the Tcl array and the data source gets set up:

  • -open — the Tcl array for the new tie gets loaded from the data source, and the previously existing contents of the Tcl array get erased. Care is taken to not erase the previous contents if creating the tie fails, which is honestly so thoughtful of it 🫶

This option and -save exclude each other. If neither this nor -save is specified, then this option is the assumed default.

  • -save — the Tcl array for the new tie gets saved to the data source, and the previously existing contents of the data source get erased. gone. deleted. not coming back 💀

This option and -open exclude each other. If neither this nor -open is specified, then -open is the assumed default.

  • -merge — using this stops any previously existing content from being erased and merges the data instead. It can be specified alongside either -open or -save, and whichever one you pair it with decides how data that exists in both the Tcl array and the data source — i.e. the duplicates — gets dealt with.
    • When used with -open, data in the data source has precedence. In other words, for duplicates, the data in the data source gets loaded into the Tcl array. 👑
    • When used with -save, data in the Tcl array has precedence. In other words, for duplicates, the data in the Tcl array gets saved into the data source. 💅
::tie::untie arrayvarname ?token?

This one dissolves one or more ties associated with the Tcl array named by arrayvarname. If you don't specify a token, then all ties to that Tcl array get dissolved. If you do, then only the tie that token stands for gets removed — but only if it's actually connected to the array. Trying to remove a specific tie that doesn't belong to the array you provided will cause an error. So don't do that 😭

heads up bestie: severing a tie destroys management information internal to the package, but the data source that was handled by the tie will not be touched — it only gets closed. Your data lives on, it's just unsupervised now 🦋

After the command returns, none of the changes made to the array get saved to the data source anymore. It's over, it's giving closure.

The result of the command is an empty string.

  • varname arrayname (in) — the name of a Tcl array variable that may have ties 🎀
  • handle token (in) — a handle representing a specific tie. This argument is optional.
::tie::info ties arrayvarname

Returns a list of ties associated with the Tcl array variable named by arrayvarname. The result list will be empty if the variable has no ties associated with it. loneliest list ever fr 😭

::tie::info types

Returns a dictionary of registered types, plus the class commands they're associated with. very "who's who" energy ✨

::tie::info type dstype

Returns the fully resolved class command for a type name. Which means it'll follow a whole chain of type definitions all the way to its end. follow the yellow brick road, bestie 🌈

🎀 STANDARD DATA SOURCE TYPES 🎀

This package ships with the six following types as examples and standard data sources. iconic lineup honestly 👑

  • log — this data source maintains no actual data and no persistence whatsoever. It accepts no identifying arguments. All changes are simply logged to stdout. it's just vibing and narrating 🗣️
  • array — this data source uses a regular Tcl array as the origin of the persistent data. It accepts a single identifying argument: the name of that Tcl array. All changes get mirrored to that array. 🪞
  • remotearray — similar to array, but with a glow-up: the Tcl array we're mirroring to isn't directly accessible — we reach it through a send-like command.

It accepts three identifying arguments, in this order: the name of the other Tcl array, the command prefix for the send-like accessor command, and an identifier for the remote entity hosting the array. All changes get mirrored to that array via the command prefix. All commands will be executed in the context of the global namespace.

"send-like" means the command prefix has to have send syntax and semantics — i.e. it's a channel over which we can send arbitrary commands to some other entity. The remote array data source, though, only ever uses the commands set, unset, array exists, array names, array set, and array get to retrieve and set values in the remote array. That's the whole toolkit, bestie 🧰

The command prefix and the entity id are kept separate so the data source can use options like -async when it's assembling the actual commands. very modular of it 💅

Examples of command prefixes, listed with the id of the remote entity, without options. In reality, only the part before the id is the command prefix:

  • send tkname — the Tcl array is in a remote interpreter and is accessed via Tk's X communication.
  • comm::comm send hostportid — the Tcl array is in a remote interpreter and is accessed through a socket.
  • thread::send threadid — the Tcl array is in a remote interpreter in a different thread of this process.
  • file — this data source uses a single file as the origin of the persistent data. It accepts a single identifying argument: the path to that file. The file has to be both readable and writable. It may not exist — the data source will create it in that case, and this (and only this) situation will require that the directory for the file exists and is writable as well. so like, know where you're putting it 🍓

All changes get saved in the file as proper Tcl commands, one command per operation. In other words, the file will always contain a proper Tcl script. periodt.

If the file already exists when the tie using it gets set up, then it gets compacted — i.e. superfluous operations get removed — if the operations log stored in it contains either at least one operation clearing the whole array, or at least 1.5 times more operations than entries in the loaded array. math said clean it up 🧚

  • growfile — this data source is like file in terms of the storage medium for the array data and how it's configured. But in contrast to the former, it assumes and ensures that the tied array will never shrink. I.e. creating new array entries and modifying existing entries is allowed, but deleting entries is not, and doing so causes the data source to throw errors. 🔥

This restriction lets us radically simplify both the file format and access to the file. For one, the file is read only once and the internal cache cannot be invalidated. Second, writing data reduces to a simple append, and no compaction step is necessary. The format of the contents is the string representation of a dictionary, which can be incrementally extended forever at the end. infinite growth, no shrinking, the dream tbh 💖

  • dsource — this data source uses an explicitly specified data source object as the source for the persistent data. It accepts a single identifying argument: the command prefix, i.e. the object command.

To use this type, you gotta know how the framework manages ties and what data source objects are. iykyk 😌

All changes get delegated to the specified object.

💖 CREATING NEW DATA SOURCES 💖

okay so real talk: if you're just using ties, this whole section is not for you, bestie. This is for the devs who wanna build new data source classes. Periodt.

🧸 DATA SOURCE OBJECTS 🧸

All ties are represented internally by an in-memory object that mediates between the tie framework and the specific data source, like an array, file, etc. This is the data source object.

Its class, the data source class, is not generic, but specific to the type of the data source. Writing a new data source requires us to write such a class, and then registering it with the framework as a new type.

The following subsections describe the various APIs a data source class and the objects it generates will have to follow to be compatible with the tie framework.

Data source objects are normally automatically created and destroyed by the framework when a tie is created, or removed. This management can be explicitly bypassed through the usage of the "dsource" type. The data source for this type is a data source object itself, and this object is outside of the scope of the tie framework and not managed by it. In other words, this type allows the creation of ties which talk to pre-existing data source objects, and these objects will survive the removal of the ties using them as well.

✨ REGISTERING A NEW DATA SOURCE CLASS ✨

After a data source class has been written it is necessary to register it as a new type with the framework.

  • ::tie::register dsclasscmd as dstype — Using this command causes the tie framework to remember the class command dsclasscmd of a data source class under the type name dstype.

After the call the argument dstype of the basic user command ::tie::tie will accept dstype as a type name and translate it internally to the appropriate class command for the creation of data source objects for the new data source.

👑 DATA SOURCE CLASS 👑

Each data source class is represented by a single command, also called the class command, or object creation command. Its syntax is:

dsclasscmd objname ?dsname...?
  • The first argument of the class command is the name of the data source object to create. The framework itself will always supply the string %AUTO%, to signal that the class command has to generate not only the object, but the object name as well.

This is followed by a series of arguments identifying the data source the new object is for. These are the same dsname arguments which are given to the basic user command ::tie::tie. Their actual meaning is dependent on the data source class.

The result of the class command has to be the fully-qualified name of the new data source object, i.e. the name of the object command. The interface this command has to follow is described in the section DATA SOURCE OBJECT API.

🛠️ DATA SOURCE OBJECT API 🛠️

Please read the section DATA SOURCE CLASS first, to know how to generate new object commands.

Each object command for a data source object has to provide at least the methods listed below for proper inter-operation with the tie framework. Note that the names of most of the methods match the subcommands of the builtin array command.

  • ds destroy — This method is called when the object ds is destroyed. It now has to release all its internal resources associated with the external data source.
  • ds names — This command has to return a list containing the names of all keys found in the data source the object talks to. This is equivalent to array names.
  • ds size — This command has to return an integer number specifying the number of keys found in the data source the object talks to. This is equivalent to array size.
  • ds get — This command has to return a dictionary containing the data found in the data source the object talks to. This is equivalent to array get.
  • ds set dict — This command takes a dictionary and adds its contents to the data source the object talks to. This is equivalent to array set.
  • ds unset ?pattern? — This command takes a pattern and removes all elements whose keys matching it from the data source. If no pattern is specified it defaults to *, causing the removal of all elements. This is nearly equivalent to array unset.
  • ds setv index value — This command has to save the value in the data source the object talks to, under the key index.

The result of the command is ignored. If an error is thrown then this error will show up as error of the set operation which caused the method call.

  • ds unsetv index — This command has to remove the value under the key index from the data source the object talks to.

The result of the command is ignored. If an error is thrown then this error will show up as error of the unset operation which caused the method call.

  • ds getv index — This command has to return the value for the key index in the data source the object talks to.

And here's a little comparison table — but make it a list, bestie:

Regular Tcl → Data source

  • array names a → ds names
  • array size a → ds size
  • array get a → ds get
  • array set a dict → ds set dict
  • array unset a pattern → ds unset ?pattern?
  • set a($idx) $val → ds setv idx val
  • unset a($idx) → ds unsetv idx
  • $a($idx) → ds getv idx

🐛 BUGS, IDEAS, FEEDBACK 🐛

This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category tie of the Tcllib Trackers http://core.tcl.tk/tcllib/reportlist. Please also report any ideas for enhancements you may have for either package and/or documentation.

When proposing code changes, please provide unified diffs, i.e the output of diff -u.

Note further that attachments are strongly preferred over inlined patches. Attachments can be made by going to the Edit form of the ticket immediately after its creation, and then using the left-most button in the secondary navigation bar.

🔑 KEYWORDS 🔑

  • array
  • database
  • file
  • metakit
  • persistence
  • tie
  • untie

📦 CATEGORY 📦

Programming tools

Copyright (c) 2004-2021 Andreas Kupries <andreas_kupries@users.sourceforge.net>

okay bestie that's the whole page! source/version: tcllib 1.3, and this is tie(3tcl). (The original footer didn't show a date, but you know the vibe 💅) stay slay, and don't forget to report those bugs! 💖✨
📦 where this page comes from