vfs(3tcl)

📦
version 1.4.2~20230906 · 2 versions
, in debian trixie, ubuntu questing, ubuntu resolute, ubuntu stonking

🌈 vfs — Tcl-only Virtual File Systems 💅

okay bestie, welcome to the vfs(3tcl) page, aka the manual for the Tcl-only Virtual File Systems situation. this is part 1 of 2, so hydrate and stay with me ✨

✨ NAME ✨

::vfs — Commands and Procedures to create virtual filesystems

🎀 SYNOPSIS 🎀

package require Tcl

package require vfs ?1.4?

vfs::filesystem info

vfs::filesystem mount

vfs::filesystem unmount

vfs::accessMode mode

vfs::matchDirectories types

vfs::matchFiles types

vfs::matchCorrectTypes types filelist ?inDir?

🦋 DESCRIPTION 🦋

so the ::vfs package is giving query, mount and unmount virtual filesystems energy, and it also hands you some Tcl libraries that help you write brand-new virtual filesystems in Tcl. no cap. 🫶

the second a virtual filesystem is in place, the standard Tcl file, glob, cd, pwd, open commands — including all their C APIs in the Tcl library (like Tcl_FSOpenFileChannel, Tcl_FSMatchInDirectory, ...) — can be used inside the filesystem. and bestie, even properly written extensions like Tk that might open or read files will transparently access the virtual filesystem too. it's giving invisible but iconic 💅

the way that this works is: because all of Tcl's FS activity passes through a single layer, it can all be intercepted. and this package does exactly that. ✨

notice that this is quite different to overloading the file command in Tcl. we are actually providing vfs replacements for C commands like access, stat. by implementing just a handful of commands at this low level, we ensure that all commands at higher levels function irrespective of what is going on inside the FS layer. main character architecture tbh 👑

Tcl's filesystem hooks operate on a per-process basis. that means every Tcl interpreter in the same process/application sees the same filesystem, including any virtual filesystems. one process, one vibe. 🌸

the package require vfs command should be used to access this library. it automatically registers the vfs hooks into Tcl's filesystem, and these will not be removed until Tcl exits (if desired, control over this could be exposed to Tcl in the future). however, the vfs package will at that stage not have any new filesystems mounted, so it will have little effect. lowkey harmless, ngl.

note that package require vfs has two effects, bestie:

  1. when it is issued in any Tcl interpreter it will ensure the vfs hooks have been registered with Tcl's core just once (and if any of those interpreters are later deleted, the vfs hooks will still remain registered — they remain until Tcl exits). iconic permanence 💀
  2. the second effect is to provide the command vfs::filesystem which allows the interpreter to intercept filesystem commands and handle them with Tcl code in that interpreter.

there are three somewhat unsupported subcommands of vfs::filesystem, namely fullynormalize path, posixerror int, internalerror ?script?, which are used to normalize a path (including any final symlink), to register a posix error code with a Tcl error, and to trap/report internal errors in tclvfs implementations respectively. they said "somewhat unsupported" and i felt that 🙈

💖 vfs::filesystem mount ?-volume? path command

to use a virtual filesystem, it must be 'mounted'. mounting involves declaring to the vfs package that any subdirectories of a given path in the filesystem should be handled by the given command, which should be a Tcl command or procedure in the interpreter in which the vfs::filesystem is executed.

if the ?-volume? flag is given, the given mount point is also registered with Tcl as a new volume (like a new drive which will appear in file volumes). this is useful (and required for reasonable operation) for mounts like ftp://. for paths mounted inside the native filesystem, it should of course not be given. 🚫

the new filesystem mounts will be observed immediately in all interpreters in the current process. and if the interpreter is later deleted, all mounts which are intercepted by it will be automatically removed (and will therefore affect the view of the filesystem seen by all interpreters). so yeah, cleanup is automatic, slay 🧹✨

🌸 vfs::filesystem unmount path

this unmounts the virtual filesystem which was mounted at path (hence removing it from Tcl's filesystem), or throws an error if no filesystem was mounted there. bye bestie 👋

⭐ vfs::filesystem info ?path?

if no arguments are given, this returns a list of all filesystems mounted (in all interpreters). if a path argument is given, then the command to be used for that path is returned, or an error is thrown if no vfs is mounted for that path.

there is currently no facility for examining in which interpreter each command will be evaluated. we just don't know her like that, sorry 🥲

💅 vfs::filesystem fullynormalize path

performs a full expansion of path, (as per file normalize), but including following any links in the last element of path. yes the last element. yes that matters. periodt. 👑

🔮 IMPLEMENTING A TCL ONLY VFS 🔮

okay so here's the whole deal, bestie. the vfs package is gonna intercept every single filesystem operation that falls inside a given mount point, and hand it straight off to that mount point's command — the one living in the interpreter that registered it. generally this goes down like the C equivalent of an eval that looks like this:

eval $command [list $subcmd $root $relative $actualpath] $args.

and subcmd can be any of these cuties: access, createdirectory, deletefile, fileattributes, matchindirectory, open, removedirectory, stat, utime. if your command handles each one of those cases properly, congrats, you have a complete, perfect virtual filesystem — genuinely indistinguishable from the native one as far as Tcl can tell. (CAVEATS: right now it does not expose to Tcl all the permission-related flags of glob. 🙈)

the rest of the arguments say which file path we're operating on (every one of these commands works on exactly one path), plus whatever extra args are needed to actually do the thing. the file path comes in three pieces:

  • root — the part of the path that lives outside this filesystem's mount point
  • relative — the part of the path that lives inside this filesystem
  • actualpath — the original, unnormalized name of the path that was used in the current command, wherever it came from (Tcl or C)

like, for example, say C:/foo/bar/mount.zip/xxx/yyy is a path in your filesystem, where mount.zip is a zip archive that's been mounted (on top of itself, iconic) and it contains xxx/yyy, and your current working directory is inside xxx, and then we evaluate something like file exists yyy. in that case root will be C:/foo/bar/mount.zip, relative will be xxx/yyy, and actualpath will be yyy. 💅 and yes, the file separator between the root and the relative is omitted.

ngl, most filesystem operations will only need the relative argument to work correctly, but the other two are genuinely required for some subcommands to behave right. so don't skip them.

almost all of these commands should either return correctly (like, with a TCL_OK result at the C level) or they should use vfs::filesystem posixerror to signal the appropriate posix error code. if a Tcl error gets thrown, treat that as a bug — though it will be interpreted as an unknown posix error in the filesystem call. the exceptions to these rules are the filesystem commands that can specify a Tcl error message directly:

  • open (when an interpreter is given)
  • matchindirectory
  • fileattributes (for a set or get operation only)

these three are allowed to throw any Tcl error message they want, which gets passed along to the caller, or they can throw a posix error which gets handled appropriately.

so the actual commands are as follows — and where you see r-r-a, that's just shorthand for the standard argument triplet of root, relative and actualpath. 🫶

command access r-r-a mode

Return TCL_OK or throw a posix error, depending on whether the given access mode (an integer) is compatible with the file.

command createdirectory r-r-a

Make a directory with the given name. the command can safely assume that all sub-directories in the path already exist and are valid, and that the actual desired path does not exist yet — Tcl takes care of all of that for us. so relaxed. 😌

command deletefile r-r-a

Delete the given file. bye bestie 💀

command fileattributes r-r-a ?index? ?value?

  • if neither index nor value is given → return a list of all acceptable attribute names
  • if index is given but no value → retrieve the value of the index'th attribute (counting in order over the list you get when no argument is given) for the given file
  • if a value is also given → set the index'th attribute of the given file to that value

command matchindirectory r-r-a pattern types

Return the list of files or directories in the given path (which is always the name of an existing directory) that match pattern and are compatible with the types given. 🎀 it is very important that the command correctly handles types requests for directories only (and files only), because to handle any kind of recursive globbing, Tcl will actually generate requests for directory-only matches straight from the filesystem. see vfs::matchDirectories below for help, it's literally right there for you.

command open r-r-a mode permissions

for this one, mode is any of "r", "w", "a", "w+", "a+". if the open involves creating a file, then permissions dictates what modes to create it with. if the open operation was not successful, an error should be thrown.

if the open operation was successful, the command should return a list of either one or two items:

  1. the first item (obligatory) — the name of the channel that's been created
  2. the second item (optional) — a Tcl-callback to be used when the channel is closed, so the vfs can clean up as appropriate

this callback will be evaluated by Tcl just before the channel is closed. the channel will still exist, and all available data will have been flushed into it. the callback can, for example, seek to the beginning of the channel, read its contents and store that contents elsewhere (e.g. compressed, or on a remote ftp site, etc). the return code or any errors returned by the callback are ignored (if the callback wants to signal an error it has to do so asynchronously, with bgerror for example) — unless the 'internalerror' script has been specified, in which case they get passed to that script for further action.

command removedirectory r-r-a recursive

Delete the given directory. recursive is either 0 or 1.

  • if it's 1, then even if the directory is non-empty, an attempt should be made to recursively delete it and its contents
  • if it's 0 and the directory is non-empty, a posix error (ENOTEMPTY) should be thrown

command stat r-r-a

Return a list of even length containing field-name and value pairs for the contents of a stat structure. the order is not important, so don't stress. ✨ the option names are:

  • dev (long)
  • ino (long)
  • mode (int)
  • nlink (long)
  • uid (long)
  • gid (long)
  • size (long)
  • atime (long)
  • mtime (long)
  • ctime (long)
  • type (string which is either "directory" or "file")

where the type of each argument is given in brackets. so the procedure should return with something like return [list dev 0 type file mtime 1234 ...].

command utime r-r-a actime mtime

Set the access and modification times of the given file (these are read with stat).

🛠️ VFS HELPERS 🛠️

the vfslib provides a bunch of Tcl procedures that can help you write command procedures to handle all of the above possibilities. here they are:

  • vfs::accessMode mode — converts an integer access mode to a somewhat more preferable string, any of F X W XW R RX RW.
  • vfs::matchDirectories types — does types want directories included?
  • vfs::matchFiles types — does types want files included?
  • vfs::matchCorrectTypes types filelist ?inDir? — returns that subset of the filelist (which are either absolute paths or names of files in inDir) which are compatible with the types given.

🐛 VFS DEBUGGING 🐛

when your implementation is being a little drama queen and you need to figure out why, use something like this:

vfs::filesystem internalerror report ; proc report {} { puts stderr $::errorInfo }

⚠️ LIMITATIONS ⚠️

ngl there are very few limitations to the vfs code. but here's the tea. 🍓

one subtlety you may run into: if you mount a case-sensitive virtual filesystem into a case-insensitive system (e.g. the standard Windows or MacOS fs) and your code relies on case-insensitivity, then it will not run properly in the virtual filesystem. of course, if your code relies on case-insensitivity, it wouldn't run under Tcl on Unix either — so the best solution is to fix your code! periodt. 💅

we may add link and lstat commands in the future to allow virtual filesystems to support reading and writing links — this is supported by the C API, but has simply not been exposed to Tcl in this extension yet. someday bestie. 🌈

the Tcl Tcl_FSMatchInDirectory function takes a variety of type information in a Tcl_GlobTypeData structure. we currently only expose the 'type' field from that structure (so the 'permissions' and MacOS type/creator fields are ignored).

🔑 KEYWORDS 🔑

vfs, filesystem, file

and that's the whole page, bestie — no cap, you now know this manual better than the person who wrote it. 🫶 this was vfs(3tcl), version 1.4, and it lives in the Tcl manual set — go forth and mount something iconic. 💖✨👑

📦 where this page comes from