RFC 1037
NFILE - a file access protocol
RFC 1037 🎀
NFILE - A File Access Protocol ✨
Okay so this one's from the Network Working Group 🫶 It was written by B. Greenberg and S. Keene, and it's dated December 1987. The header doesn't say what category this is or whether it obsoletes or updates anything, so I'm not guessing 🙈
STATUS OF THIS MEMO 📝
Bestie, this doc specs out NFILE, plus the two layers it sits on: the Token List Transport Layer and Byte Stream with Mark 💅 The whole point is to get people talking about these ideas and to push designers of future file protocols to use them. There's also a secondary goal: making the spec easy to get for any site that could benefit from implementing NFILE. And the distribution is unlimited, so share away 💖
TABLE OF CONTENTS 📚
TABLE OF CONTENTS
Page
1. INTRODUCTION 3
2. NFILE PROTOCOL LAYERING 4
3. OVERVIEW OF AN NFILE SESSION 5
4. NFILE CONTROL AND DATA CONNECTIONS 6
5. NFILE FILE OPENING MODES 7
6. NFILE CHARACTER SET 9
7. CONVENTIONS USED IN THIS DOCUMENT 10
7.1 Mapping Data Types Into Token List Representation 10
7.2 Format of NFILE Commands and Responses 10
7.3 Data Channel Handles and Direct File Identifiers 13
7.4 Syntax of File and Directory Pathname Arguments 13
7.5 Format of NFILE File Property/Value Pairs 14
8. NFILE COMMANDS 16
8.1 ABORT Command 16
8.2 CHANGE-PROPERTIES Command 16
8.3 CLOSE Command 17
8.4 COMPLETE Command 19
8.5 CONTINUE Command 20
8.6 CREATE-DIRECTORY Command 21
8.7 CREATE-LINK Command 21
8.8 DATA-CONNECTION Command 22
8.9 DELETE Command 23
8.10 DIRECT-OUTPUT Command 23
8.11 DIRECTORY Command 24
8.11.1 NFILE DIRECTORY Data Format 26
8.12 DISABLE-CAPABILITIES Command 27
8.13 ENABLE-CAPABILITIES Command 28
8.14 EXPUNGE Command 28
8.15 FILEPOS Command 29
8.15.1 Implementation Hint for FILEPOS Command 30
8.16 FINISH Command 30
8.17 HOME-DIRECTORY Command 31
8.18 LOGIN Command 32
8.19 MULTIPLE-FILE-PLISTS Command 34
8.20 OPEN Command 35
8.20.1 NFILE OPEN Optional Keyword/Value Pairs 39
8.20.2 NFILE OPEN Response Return Values 45
8.21 PROPERTIES Command 47
8.22 READ Command 49
8.23 RENAME Command 50
8.24 RESYNCHRONIZE-DATA-CHANNEL Command 51
8.24.1 Implementation Hints for RESYNCHRONIZE-DATA- 51
CHANNEL Command
8.25 UNDATA-CONNECTION Command 52
9. NFILE RESYNCHRONIZATION PROCEDURE 53
9.1 NFILE Control Connection Resynchronization 54
9.2 NFILE Data Connection Resynchronization 55
10. NFILE ERRORS AND NOTIFICATIONS 58
10.1 Notifications From the NFILE Server 58
10.2 NFILE Command Response Errors 59
10.3 NFILE Asynchronous Errors 60
10.4 NFILE Three-letter Error Codes 61
11. TOKEN LIST TRANSPORT LAYER 65
11.1 Introduction to the Token List Transport Layer 65
11.2 Token List Stream 66
11.2.1 Types of Tokens and Token Lists 66
11.2.2 Token List Stream Example 68
11.2.3 Mapping of Lisp Objects to Token List Stream 70
Representation
11.2.4 Aborting and the Token List Stream 71
11.3 Token List Data Stream 72
12. BYTE STREAM WITH MARK 73
12.1 Discussion of Byte Stream with Mark 73
12.2 Byte Stream with Mark Abortable States 75
13. POSSIBLE FUTURE EXTENSIONS 77
APPENDIX A. NORMAL TRANSLATION MODE 79
APPENDIX B. RAW TRANSLATION MODE 83
APPENDIX C. SUPER-IMAGE TRANSLATION MODE 84
NOTES 86
LIST OF TABLES
TABLE 1. TRANSLATIONS FROM NFILE CHARACTERS TO UNIX CHARACTERS 80
TABLE 2. TRANSLATIONS FROM UNIX CHARACTERS TO NFILE CHARACTERS 80
TABLE 3. TRANSLATIONS FROM NFILE TO PDP-10 CHARACTERS 81
TABLE 4. TRANSLATIONS FROM PDP-10 CHARACTERS TO NFILE 82
CHARACTERS
TABLE 5. SUPER-IMAGE TRANSLATION FROM NFILE TO ASCII 84
TABLE 6. SUPER-IMAGE TRANSLATION FROM ASCII TO NFILE 851. INTRODUCTION ✨
Okay so NFILE stands for "New File Protocol" 💅 It was first built as a replacement for an older protocol called QFILE, and the whole goal was fixing QFILE's robustness problems. That's where the "new" comes from.
NFILE was designed and implemented at Symbolics by Bernard S. Greenberg. Mike McMahon also made huge contributions, especially on the design and implementation of the Byte Stream with Mark and Token List Transport layers 🙌 Symbolics computers have been using NFILE for file access between themselves since 1985, and there are NFILE servers for UNIX hosts too. It's meant for any kind of file system, not just Symbolics' native one. Iconic behavior 👑
NFILE is a file access protocol that handles a big set of operations on files and directories on remote systems. That includes:
- Reading and writing entire files
- Reading and writing selected portions of files
- Deleting and renaming files
- Creating links
- Listing, creating, and expunging directories
- Listing and changing the properties of files
- Enabling and disabling access capabilities on a remote host
NFILE supports transferring binary or character files 🌸
If something goes wrong during a command, the NFILE server tells you about the error 🚨 On top of that, NFILE has a really robust way of handling aborts on the user side. Main character energy, literally 💖
This spec defines NFILE user version 2 and server version 2. The team doesn't see NFILE as a protocol that stays frozen forever 🦋 They want it to keep growing as new requirements show up through this RFC process, and the design leaves room for useful extensions. Some of the ones they're considering are described later in the doc. Check out the section "Possible Future Extensions", section 13 🧚
2. NFILE PROTOCOL LAYERING 🥞
Bestie, NFILE is a layered protocol 🎂 Here's the stack, from top to bottom:
+-----------------------------------------------+
|client program or user interface |
+-----------------------------------------------+
|NFILE |
+-----------------------------------------------+
|Token List Transport Layer |
+-----------------------------------------------+
|Byte Stream with Mark |
+-----------------------------------------------+
|reliable host-host byte transmission protocol |
+-----------------------------------------------+So Byte Stream with Mark is a simple protocol that guarantees an out-of-band signal can get sent when a program gets interrupted 🛑 It gets layered on top of existing byte stream protocols. A big goal of the NFILE design was that NFILE could run on any byte stream, and right now it's implemented on TCP and Chaosnet 🌈 Check out the section "Byte Stream with Mark", section 12.
The Token List Transport Layer is the protocol that makes it easier to send simple structured data, like lists 📜 See the section "Token List Transport Layer", section 11.
NFILE commands and command responses get sent as "token lists" ✨ See the section "NFILE Commands", section 8.
Heads up: this spec doesn't include a client program or user interface for the protocol. In the Symbolics implementation, normal file operations quietly call NFILE whenever the remote host is known to support it 🫶 Another option would be a dedicated client program that sends NFILE commands whenever the user explicitly asks for something.
(Part 1 of 17. More parts are coming, so stay tuned bestie 💜)
3. OVERVIEW OF AN NFILE SESSION ✨
okay so an NFILE session is basically a dialogue between two hosts 💖 the host that starts it is the "user side", and the other one is the "server side" 🫶 the user side is the one that sends all the NFILE commands. the server receives each command, processes it, and responds to say whether it worked or not 💅
the user side keeps track of what it sent and what came back using "transaction identifiers" 🎀 for every command, it generates a transaction identifier that must be unique for this dialogue, then sends it to the server along with the command. every server response includes the transaction identifier of the command it's answering, so the user side knows which command each response belongs to 🦋 the server is not required to respond in the same order the commands were given, so it's giving out-of-order energy 🙈
NFILE commands go over a bidirectional network connection called the "control connection" 🌸 the server sends its command responses back on that very same control connection. the control connection for the session gets set up at the very beginning. if it ever breaks, the NFILE session is over, periodt 💀
commands and responses ride the control connection, but the actual file data travels over "data channels" 🦋 an "input data channel" moves data from server to user, and an "output data channel" moves data from user to server. each input data channel is paired with an output data channel, and together those two make a "data connection" 🧚
more than one NFILE activity can be happening at any given time, bestie 🌈 for example, several files can be open and transferring simultaneously, one or more commands can be in process at the same time, and the server can be sending directory lists while it's still processing more commands. this happens in an implementation where the user side has multiple processes and several processes share a single NFILE server 👑
4. NFILE CONTROL AND DATA CONNECTIONS ✨
the user and server talk through a single control connection plus a set of data connections 💜 data connections get established and disestablished by NFILE commands. the user side sends its commands to the server over the control connection, and the server responds to every user command over that same control connection. the actual file data gets sent over the data connections 🎀
user aborts can mess with the normal flow of data on the control connection and the data connections 😭 an important part of any file protocol is how it handles user aborts, and NFILE uses a resynchronization procedure to bring the affected control connection or data channel from an unknown, unsafe state back into a known state 🫧 after that, the control connection or data channel can be reused. see the section "NFILE Resynchronization Procedure", section 9.
THE CONTROL CONNECTION 🎀
an NFILE session begins when the NFILE user program connects to a remote host and establishes a network connection 🧚 that initial connection is the control connection of the dialogue. if TCP is the underlying protocol, contact NFILE's well-known port, 59. if Chaos is used, use the contact name "NFILE".
the control connection is the vehicle the user uses to send its commands, and the server uses it to send its command responses. these types of communication happen over the NFILE control connection:
- The user side sends NFILE commands.
- The server sends command responses.
- The server sends "notifications" and "asynchronous errors". See the section "NFILE Errors and Notifications", section 10.
- During resynchronization (a special circumstance), either the user or the server sends a mark.
all commands, command responses, and other data flowing over the NFILE control connection get transmitted in the format of "top-level token lists" 📝 the control connection expects to never receive "loose tokens", meaning tokens that aren't contained in token lists 🙅♀️
DATA CONNECTIONS 🔗
data connections get established and discarded at user request, using two NFILE commands: DATA-CONNECTION and UNDATA-CONNECTION 🎶 each data connection is associated with one specific control connection, the same control connection that caused it to be established.
each data connection is made up of two data channels, and each data channel can send data in one direction. the "input channel" is the data channel that carries data from the server to the user side, and the "output channel" is the one that carries data from the user to the server side. all through the NFILE docs, "input channel" and "output channel" are seen from the perspective of the user side 👀 a single data channel can be reused for one data transfer after another.
the format of data sent over data channels is defined as a "token list data stream". see the section "Token List Data Stream", section 11.3. when the end of the data is reached, the keyword token EOF gets sent 🏁 every so often, token lists get sent over the data channels too, like asynchronous error descriptions.
5. NFILE FILE OPENING MODES ✨
the NFILE OPEN command opens a file for reading, writing, or "direct access" at the server host 📂 in general, that means asking the host file system to access the file and getting back a file number, pointer, or other quantity for quick access to the file later. this is called an "opening" 🔑 the term "opening" translates to a file stream in Symbolics terminology, a JFN in TOPS-20 terminology, and a file descriptor in UNIX terminology. an opening usually keeps track of how many bytes have been read or written, plus other bookkeeping info.
NFILE supports two ways of transferring file data, "data stream mode" and "direct access mode" 🌊 a single mode is associated with each opening. an NFILE dialogue can have more than one opening, so it can use both modes in one go.
DATA STREAM MODE 🌊
data stream mode is the default mode of NFILE's OPEN command 💖 it's appropriate when the entire file is transferred, either from user to server or from server to user. it gets used more often than direct access mode.
the OPEN command includes a "handle" argument, which identifies the data channel to be used for transferring the data. the handle is used in later commands to reference this particular opening. when a data stream opening is requested with OPEN, the file is opened and the data begins to flow immediately ⚡
the sending side transmits the entire contents of the specified file over the specified data channel as rapidly as the network permits 🚀 when the sending side reaches the end of the file, it sends a special control token to signal end of file. the receiving side expects an uninterrupted stream of bytes to appear immediately on its side of the data channel.
the user gives the CLOSE command to end a data stream transfer, and CLOSE results in closing the file 🔒
DIRECT ACCESS MODE 🎯
direct access mode lets you read or write data from a given starting point in a file, through a specified number of bytes. in this mode, data is requested and sent in individual transactions 🧩 to request a direct access mode opening, the OPEN command is used with a DIRECT-FILE-ID argument. (in data stream mode, no DIRECT-FILE-ID is supplied.) the direct file identifier is used in later commands to reference the direct access opening.
when a file is opened in direct access mode, the flow of data does not start right away 🙅♀️ instead, the user gives either a READ command (to request data flowing from server to user) or a DIRECT-OUTPUT command (to request data flowing from user to server). when reading, the READ command lets the user specify the starting point and the number of bytes to transfer. when writing, the FILEPOS command can be used to specify the starting point, before the DIRECT-OUTPUT command is given. the user can give many READ and DIRECT-OUTPUT commands, one after another 🔁
the user side ends the direct access transfer with the CLOSE command. the ABORT command can be given to stop without transmitting all of the specified bytes 🛑
6. NFILE CHARACTER SET ✨
the NFILE character set <1> is an extension of standard ASCII 🔤 NFILE command and response names only use the standard ASCII characters. however, the protocol supports transferring the non-ASCII characters in the NFILE character set, and those characters might be stored in files or used in pathnames 🌏
servers on machines that don't natively use the NFILE character set must perform character set translations for character openings, depending on the requested translation mode. no translation is required for binary openings. there are three translation modes for character openings: NORMAL, RAW, and SUPER-IMAGE. each mode specifies a way to translate between the server's native set and the NFILE character set 🗺️
NORMAL is the default mode of the OPEN command 💅 the translation for NORMAL mode makes sure that a file containing characters from the NFILE character set can be written to a remote host and read back intact. OPEN has optional keyword arguments to specify RAW or SUPER-IMAGE. RAW means perform no translation whatsoever. SUPER-IMAGE mode is intended for use by PDP-10 family machines only, and it's included largely as an illustration of a system-dependent extension 🖥️
the details of each translation mode are given in the Appendices:
- See the section "NORMAL Translation Mode", Appendix A.
- See the section "RAW Translation Mode", Appendix B.
- See the section "SUPER-IMAGE Translation Mode", Appendix C.
⚠️ position reckoning gets messy here, bestie. Some NFILE characters expand to more than one native character on some servers. so for character files, when we talk about a given position in a file or the length of a file, we have to say whether we mean "NFILE units" or "server units", because the counting of characters is different 😭
this causes major problems for file position reckoning in character files. specifically, it is futile for a user side to carefully monitor file position during output by counting characters when character translation is in effect 🙈 the server's operating system interface for "position to point x in a file" necessarily operates in server units, but the user side has counted in NFILE units. the user side can't try to second-guess the translation-counting process without losing host-independence. see the section "FILEPOS NFILE Command".
7. CONVENTIONS USED IN THIS DOCUMENT ✨
7.1. Mapping Data Types Into Token List Representation ✨
okay so throughout this NFILE spec, we keep describing arguments, return values, async error descriptions, and notifications as strings, integers, dates, time intervals, and so on 💖 but on the wire each of those conceptual types has to get mapped into the right token list representation first. Here's the official glow-up table 🎀
Conceptual Type Token List Representation
----------------------------------------------------------------
Keyword A keyword token
Keyword list A token list of keyword tokens
Integer A numeric data token
String A data token containing the characters of the
string in the NFILE character set.
Boolean Truth The token known as BOOLEAN-TRUTH.
Boolean False The empty token list.
Date A numeric data token. The date is expressed in
Universal Time format, which measures a time as
the number of seconds since January 1, 1900, at
midnight GMT.
Date-or-never Can be either a date or the empty token list,
representing "never". "Never" is used for
values such as the time a directory was last
expunged, if it has never been expunged.
Time interval A numeric data token. The time interval is
expressed in seconds. A time interval
indicating "never" is represented by the empty
token list.7.2. Format of NFILE Commands and Responses 💅
every command description starts with its command format and its response format 🧚 here's how the DATA-CONNECTION one kicks off:
Command: (DATA-CONNECTION tid new-input-handle new-output-handle)
Response: (DATA-CONNECTION tid connection-identifier)and these are the conventions every command description follows 👑
- NFILE commands and responses are transmitted as top-level token lists. 🦋 the parentheses you see in these descriptions are a shorthand, they're not sent literally over the control or data connections. the real thing is special control tokens:
TOP-LEVEL-LIST-BEGINstarts a top-level token list andTOP-LEVEL-LIST-ENDends one 🫶 - NFILE command names are keywords. 🌸 the command name is required in every command and command response, and every command name is a keyword. keywords show up in the docs as their names in uppercase, so
DATA-CONNECTIONandDELETEare two command names ✨ - A unique transaction identifier (tid) identifies each command. 💜 the tid is a string the user side makes up to identify one particular transaction (the command plus its matching response). it's abbreviated as tid in the descriptions, it's capped at fifteen characters, and it's required in every command and command response 🔥
OPTIONAL ARGUMENTS ✨
lots of NFILE commands have "optional arguments" 🌈 you can supply them with values or leave them out. if you leave one out, you have to make that explicit by putting the empty token list in its spot 🙈 the only exception is trailing optional arguments or return values, which can just be dropped without the empty token list.
for example, the DELETE description says you have to supply either a handle or a pathname, but not both, so one of them counts as optional. here's the DELETE command format 💅
(DELETE tid handle pathname)supply a handle and no pathname:
(DELETE tid handle)supply a pathname and no handle:
(DELETE tid empty-token-list pathname)in the token list stream, the empty token list looks like LIST-BEGIN immediately followed by LIST-END 🍓
OPTIONAL KEYWORD/VALUE PAIRS 🎀
four NFILE commands have "optional keyword/value pairs": COMPLETE, LOGIN, OPEN, and READ 💖 these pairs can be included or left out entirely. unlike optional arguments, you don't swap in the empty token list for a missing keyword, and the order of the pairs doesn't matter 🫶
when they're included, they're a run of alternating keywords and values. the values can be keywords, lists, strings, Booleans, integers, dates, date-or-never's, and time intervals, and each command description tells you which type fits each keyword 🌸
in the text, these pairs just show up as the keyword in uppercase. here's the LOGIN format 😭
Command Format:
(LOGIN tid user password FILE-SYSTEM USER-VERSION)FILE-SYSTEM and USER-VERSION are both optional keywords for LOGIN. the user side can send USER-VERSION and skip FILE-SYSTEM, like this 🧚
(LOGIN x105 tjones let-me-in USER-VERSION 2)a supplied pair is just the keyword followed by its value, so in that example the value for USER-VERSION is 2 💜
7.3. Data Channel Handles and Direct File Identifiers 🦋
a bunch of NFILE commands need an argument that names an opening, and in the command descriptions that argument is called a handle 🔥 it's always a string. a handle is either a data channel handle or a direct file identifier, and which one depends on the mode of the opening 👑
Data Stream 🌊
the handle must identify a data channel that's bound to an opening.
Direct Access 🎯
in general, the handle must be a direct file identifier. a direct file identifier names a direct access opening, and it's the same value you supply in the DIRECT-FILE-ID keyword/value pair in the OPEN command 🌸 it's used for every operation that identifies an opening instead of a data channel ✨
Two NFILE commands are exceptions to that general rule! 😭 the handle supplied in ABORT and CONTINUE cannot be a direct file identifier. it has to be a data channel handle instead 💀
7.4. Syntax of File and Directory Pathname Arguments 🗺️
some arguments and return values are file pathnames 🌈 those are strings written in the pathname syntax native to the server host, and they never contain host identifiers of any kind 🙅♀️ they must be fully defaulted, meaning they have a directory and a file name (plus a file type, if the server OS supports those). a device gets referenced in the pathname when that's appropriate 🧚 and if the server file system supports version numbers, there's always an explicit version number, even if that number (or other spec) is the system's own way of saying "newest" or "oldest" 💅
here are some pathname examples across different server hosts 💖
Server Host Example of File Pathname
------------------------------------------------------------
UNIX /usr/max/life.c
TOPS-20 ps:<max>life.bin.17
VMS MACD:[MAX]LIFE.FOR;3
Symbolics LMFS >max>life.lisp.newest
------------------------------------------------------------the CREATE-DIRECTORY and HOME-DIRECTORY commands take a directory as an argument 🎀 in NFILE commands, a directory is a string that names it. usually that string is in the server host's native syntax, but sometimes the native format gets tweaked a little so it's obvious the string names a directory and not a file 🦋 for example, a UNIX directory is written "/usr/max/" (with the trailing slash), not "/usr/max" 🌸
here are some directory examples for different server hosts 🫶
Server Host Example of Directory Pathname
------------------------------------------------------------
UNIX /usr/max/
TOPS-20 <max>
VMS MACD:[MAX]
Symbolics LMFS >max>hacks>
------------------------------------------------------------7.5. Format of NFILE File Property/Value Pairs
okay bestie, so a bunch of NFILE commands want info about the properties of files or directories 💅 The ones that do this are DIRECTORY, MULTIPLE-FILE-PLISTS, PROPERTIES, and CHANGE-PROPERTIES. This section explains how all that property info gets sent over the token list stream 🧚
File property info is usually sent as property/value pairs. The property says which property you mean, and the value right after it is that property's value for the file you asked about. It's giving organized 🎀
Each property is written either as a keyword or as an integer, and you can mix the two in one description, no cap 🌈 An integer is just an index into the Property Index Table, which is an array of property keywords. The server can optionally send a Property Index Table to the user while LOGIN is running, but it doesn't have to. Using integers greatly cuts down how long the transmissions are, which is so smart ✨
Now the one rule to remember 🙈 In command arguments, file properties can't be given as integers. You have to use keywords there. Integers only show up for file properties in command responses.
Here's the list of keywords for file properties. It's not meant to be restrictive, though! If a programmer implementing NFILE needs a new keyword, they can invent one that isn't on this list. Any new keyword's value is string by default. The keywords are sorted below by their conceptual data type 🍓
Data type Keywords denoting file properties
----------------------------------------------------------------
Integers BLOCK-SIZE, BYTE-SIZE, GENERATION-RETENTION-COUNT,
LENGTH-IN-BLOCKS, LENGTH-IN-BYTES,
DEFAULT-GENERATION-RETENTION-COUNT
Dates CREATION-DATE, MODIFICATION-DATE
Date-or-never's REFERENCE-DATE, INCREMENTAL-DUMP-DATE,
COMPLETE-DUMP-DATE, DATE-LAST-EXPUNGED,
EXPIRATION-DATE
Time intervals AUTO-EXPUNGE-INTERVAL
Keyword Lists SETTABLE-PROPERTIES, LINK-TRANSPARENCIES,
DEFAULT-LINK-TRANSPARENCIES
Boolean values DELETED, DONT-DELETE, DONT-DUMP, DONT-REAP,
SUPERSEDE-PROTECT, NOT-BACKED-UP, OFFLINE,
TEMPORARY, CHARACTERS, DIRECTORY
Strings ACCOUNT, AUTHOR, LINK-TO, PHYSICAL-VOLUME,
PROTECTION, VOLUME-NAME, PACK-NUMBER, READER,
DISK-SPACE-DESCRIPTION, and any keywords not
on this listThese keyword names are meant to hint at what the properties actually mean 👀 For the scoop on CREATION-DATE, check out the section "NFILE OPEN Response Return Values", section 8.20.2. And the "Reference Guide to Streams, Files, and I/O" in the Symbolics documentation set spells out the semantics Symbolics gives these properties. Go read it, you won't regret it 📚💜
8. NFILE COMMANDS
It's super important to get the conventions behind each command description, so read these first 🫶 See the section "Conventions Used in This Document", section 7.
8.1. ABORT Command
Command: (ABORT tid input-handle)
Response: (ABORT tid)ABORT cleanly interrupts and prematurely ends one direct access mode data transfer that was started with READ 🛑 The required input-handle string argument names a data channel where an input transfer is happening right now, and that transfer has to be a direct access one. input-handle has to name a data channel. It can't be a direct file identifier, bestie 🙅♀️
When the server gets the ABORT command, it checks whether a transfer is still active on that channel. If one is, the server ends it by telling the data connection logical process to stop moving bytes of data. The user side only needs to send this command when there are outstanding unread bytes. That doesn't cover the case where the user side has already disestablished or reallocated the data channel 🙈
Whether or not a transfer is active on that channel, the user side puts the data channel into the unsafe state 🚫 Before anyone can use that data channel again, it has to be resynchronized. No skipping this step!
8.2. CHANGE-PROPERTIES Command
Command: (CHANGE-PROPERTIES tid handle pathname property-pairs)
Response: (CHANGE-PROPERTIES tid)CHANGE-PROPERTIES changes one or more properties of a file, and it's giving glow-up energy ✨ You have to give either a handle or a pathname, but not both. Whichever one you give has to be a string. handle identifies a data channel that's bound to an open file, and it can be a direct file identifier. pathname identifies a file on the server machine 🧚
property-pairs is a required token list of keyword/value pairs. The keyword is the name of the property you want to change, and the value is the new value you want it to have 💖
Which properties can be changed depends on the host, and so do any restrictions on what values those properties can take 🌸 The properties you can change are the same ones returned as settable-properties in the command response for the PROPERTIES command, so check there first.
The server tries to modify every property listed in property-pairs to the new values you asked for. Right now there's no definition for what happens if the server manages to change some of the properties but not others 🤷♀️
For more on file property keywords and their values, see the section "Format of NFILE File Property/Value Pairs", section 7.5.
8.3. CLOSE Command
Command: (CLOSE tid handle abort-p)
Response: (CLOSE tid truename binary-p other-properties)CLOSE ends a data transfer and frees up a data channel 🦋 The handle has to be a data channel handle for a data stream opening, or a direct file identifier for a direct access opening. If you give a data channel, a transfer must be active on that handle. If abort-p is supplied as Boolean truth, the file gets close-aborted, which I'll explain in a sec 👀
"Closing the file" means different things on each operating system, so it's kind of a vibe check 🌈 In general it means the pointer or logical identifier the operating system gave you when the file was "opened" becomes invalid, and the operating system and/or job resources tied to the active file access get freed up. For output files, it means making sure every last bit the user sent has actually been written to disk 💾 The server should not send a successful response until all of those things have finished successfully.
In either data stream or direct access mode, the user can ask the server to close-abort the file, instead of just closing it normally 😮💨 To close-abort a file means closing it in a way that, if possible, makes it look like the file was never opened at all. If the file was being created, it has to look like it was never created. This might be harder to pull off on some operating systems than others, but tricks with temporary names and close-time renamings by the server can usually get it done. If the file was being appended to, close-abort means forgetting the appended data 🫠
AN UNSUCCESSFUL CLOSE OPERATION
For a normal CLOSE (not a close-abort), after writing every last bit the user sent to disk and before closing the file, the server checks the data channel specified by handle to see if an asynchronous error is outstanding on that channel. In other words, the server has to figure out whether it sent an asynchronous error description to the user that the user hasn't answered yet with a CONTINUE command. If so, the server can't close the file, so it sends a command error response saying an error is pending on the channel. The right three-letter error code here is EPC. See the section "NFILE Errors and Notifications", section 10 💅
A SUCCESSFUL CLOSE OPERATION
The return values for OPEN and CLOSE are syntactically identical, but the values can change between when the file gets opened and when it gets closed 🦋 For example, the truename return value is supplied after all the close-time renaming of output files is done and the version numbers are resolved (for operating systems that support version numbers). So on some systems, a file's truename can be one thing when it's opened and something totally different once it's closed 😭 For a description of the CLOSE return values, see the section "NFILE OPEN Response Return Values", section 8.20.2.
If the user gives the CLOSE command with abort-p supplied as Boolean truth, which means they're asking for a close-abort, the server doesn't need to check whether an asynchronous error description is outstanding on the channel. It just close-aborts the file, no questions asked 🧚♀️✨
8.4. COMPLETE Command ✨
Command: (COMPLETE tid string pathname DIRECTION NEW-OK DELETED)
Response: (COMPLETE tid new-string success)Okay so COMPLETE is the command that does file pathname completion 🎀. You know when you start typing a filename and hit tab? This is the remote version of that. 💅
The string is the partial filename the user has typed, and pathname is the default name it's being typed against. Both are required, and both are strings. Everything after that is optional keyword/value pairs. 🦋
NEW-OK is a Boolean. If it's followed by Boolean truth, the server should allow either a file that already exists or one that doesn't exist yet. The default is false, which means the server ignores files that don't already exist. Not me being picky about the defaults 🙈
DELETED is also a Boolean. If it's followed by truth, the server looks for files that were deleted but not yet expunged, along with the regular non-deleted ones. By default it ignores soft-deleted files. 🗑️
DIRECTION can be followed by READ if the file is going to be read, or WRITE if it's going to be written. READ is the default. 📖
The filename gets completed based on the files actually sitting on the host file system, and you get back new-string, the expanded string. It's always a string containing a file name. It's either your original string or a new, more specific one, which is giving very "narrowing it down" energy ✨
The success value tells you how the completion went. OLD or NEW means full success. An empty token list means failure. Here are the possible values:
Value Meaning
----------------------------------------------------------------
OLD Success: the string completed to the name of
a file that exists.
NEW Success: the string completed to the name of
a file that could be created.
Empty token list Failure due to one of these reasons:
The file is on a file system that does not
support completion. new-string is supplied as
the unchanged string.
There is no possible completion. new-string
is supplied as the unchanged string.
There is more than one possible completion.
The given string is completed up to the first
point of ambiguity, and the result is supplied
as new-string.
A directory name was completed. Completion
was not successful because additional
components to the right of this directory
remain to be specified. The string is
completed through the directory name and the
delimiter that follows it, and the result is
returned in new-string.The semantics of COMPLETE aren't written down here. If you want the recommended semantics, go check the "Reference Guide to Streams, Files, and I/O" in the Symbolics documentation set. 📚🌸
8.5. CONTINUE Command ▶️
Command: (CONTINUE tid handle)
Response: (CONTINUE tid)CONTINUE picks up a data transfer that got paused because of an asynchronous error. 🔄 Every asynchronous error description can have an optional RESTARTABLE argument, which says whether it even makes sense to try again after that particular error. CONTINUE only tries to resume if the error is potentially recoverable, based on that RESTARTABLE argument. If you want the full scoop on asynchronous errors, check out the section "NFILE Errors and Notifications", which is section 10. 🧐
handle is a required string that points to the data channel that got hit with the asynchronous error. That channel could've been carrying a data stream or a direct access transfer. Also, handle cannot be a direct file identifier, so don't even try 🙅♀️
If the asynchronous error description doesn't include RESTARTABLE, and the user issues CONTINUE anyway, the server sends back a command error response. ❌ Like, it's not gonna work, babe.
8.6. CREATE-DIRECTORY Command 📁
Command: (CREATE-DIRECTORY tid pathname property-pairs)
Response: (CREATE-DIRECTORY tid dir-truename)CREATE-DIRECTORY makes a directory on the remote file system. 🏗️ The required pathname argument is a string with the pathname of the directory you want to create. The dir-truename that comes back is the pathname of the directory that was actually created. Both of these are directory pathnames, so check the section "Syntax of File and Directory Pathname Arguments", which is section 7.4. 🗺️
property-pairs is a keyword/value list of properties that define more about the directory's attributes. The allowed keywords and values depend on the operating system. Usually they're the arguments you'd hand to the native primitive for making directories. 🧩
If property-pairs is the empty token list, default access and creation attributes apply, and the server should make sure those are in place. For more on this, see the section "Format of NFILE File Property/Value Pairs", which is section 7.5. 💖
8.7. CREATE-LINK Command 🔗
Command: (CREATE-LINK tid pathname target-pathname properties)
Response: (CREATE-LINK tid link-truename)CREATE-LINK makes a link on the remote file system. 💜
pathname is the pathname of the link you're creating, and target-pathname is the spot in the file system the link points to. Both are required. The link-truename that comes back names the link that got created. ✨
If a server on a file system that doesn't support links gets a CREATE-LINK command, it sends back a command error response. 💀 No links for you, sorry.
The pathname and target-pathname arguments, plus the link-truename return value, are all strings in the full pathname syntax of the server host. See the section "Syntax of File and Directory Pathname Arguments", section 7.4, for the deets. 🗂️
properties is a required token list of keyword/value pairs. These properties and their values set certain attributes for the link. The allowed keywords and values depend on the operating system, and usually they're the arguments for the native primitive that creates links. 🧚
If no property pairs are given in the command, the server should apply a reasonable default set of attributes to the link. Check the section "Format of NFILE File Property/Value Pairs", section 7.5, if you want more info. 🎀
8.8. DATA-CONNECTION Command 🔌
Command: (DATA-CONNECTION tid new-input-handle new-output-handle)
Response: (DATA-CONNECTION tid connection-identifier)DATA-CONNECTION lets the user side kick off the setup of a brand new data connection. 🚀 The user side provides two required string arguments, new-input-handle and new-output-handle. Later commands use these to reference the two data channels that make up the connection being created. new-input-handle describes the server-to-user data channel, and new-output-handle describes the user-to-server channel. Neither of them can refer to a data channel that's already in use. 🙅♀️
Once the server gets DATA-CONNECTION, it sets up a logical port (called a socket or contact name on some networks) and makes it available on the foreign host machine. Once that port is ready, the server must tell the user what it is. It does this in the command response, through the required connection-identifier, which is a string. Then the server listens on the port named by connection-identifier and waits for the user side to connect. 👂
When the user side gets that successful command response, it hands the connection-identifier to its local network implementation so it can connect to the specified port. The data connection isn't fully established until the user side actually connects to that port. This command is kind of unusual: a successful response doesn't mean the command is finished. It only means the server has done its part of setting up the data connection. 🤯
The connection-identifier tells the user exactly which logical port the server provided. NFILE expects it to be a string. For TCP, it's the port number written in decimal. For Chaosnet, it's the contact name. The connection-identifier only gets used once. In every later NFILE command that needs either data channel from this connection, you use new-input-handle and new-output-handle instead. 🦋
For background info, check the section "NFILE Control and Data Connections", which is section 4. 📖
8.9. DELETE Command 🗑️
Command: (DELETE tid handle pathname)
Response: (DELETE tid)DELETE deletes a file on the remote file system. 💥
You have to supply either a handle or a pathname, but not both. If you give a handle, it must be a data channel handle for a data stream opening, or a direct file identifier for a direct access opening. pathname is a string in the full pathname syntax of the server host. See the section "Syntax of File and Directory Pathname Arguments", section 7.4. 🗺️
When you give a pathname, DELETE deletes that specified file. The result depends on the operating system, though. DELETE does soft deletion on TOPS-20 and LMFS, and hard deletion on UNIX and Multics. So what "deleted" means really depends on where you are 🤷♀️ If someone tries to delete a delete-through link on a Symbolics LMFS, its target gets deleted instead. Plot twist 🎭
If you supply the handle argument to DELETE, the server deletes the open file tied to the data channel named by handle, but it waits until close time to do it. This is true for both output and input. ⏳
8.10. DIRECT-OUTPUT Command 🌊
Command: (DIRECT-OUTPUT tid direct-handle output-handle)
Response: (DIRECT-OUTPUT tid)DIRECT-OUTPUT starts and stops output data flow for a direct access file opening. 🚦 It's the command that explicitly controls binding and unbinding an output data channel to a direct access opening. 🔗
direct-handle is a required argument, and output-handle is optional. 🎀
If you supply output-handle, that's a request to bind an output data channel (the one named by output-handle) to the direct access opening designated by direct-handle. The output data channel you name has to be free. Once it's bound, the server starts accepting data from that connection and writing it to the opening. 🌸
If you leave output-handle out, that's a request to unbind the channel and end the active output transfer. Bye-bye, data flow 👋
8.11 DIRECTORY Command 📂
Command: (DIRECTORY tid input-handle pathname control-keywords
properties)
Response: (DIRECTORY tid)DIRECTORY returns a directory listing with the identities and attributes for logically related groups of files, directories, and links. 🗂️ If the command succeeds, one single token list containing the requested info gets sent over the data channel named by input-handle, and then both sides implicitly free that data channel <2>. For the details on the token list format, see the section "NFILE DIRECTORY Data Format", section 8.11.1. 💜
pathname says which files to describe, and it's a string in the full pathname syntax of the server host. ✨ See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
The pathname usually has wildcard characters in it, in the operating system's own format, describing the possible file name matches. 🌸 Most operating systems have a facility that takes a pathname like that and returns info about every matching file. Some operating systems let wildcards match multiple things in the directory or device parts of the pathname, and others don't. 🙈 There's no clear contract yet about what servers on systems that don't allow wildcard matches (or some kinds of wildcard matches) are supposed to do when they get a wildcard.
Heads up: this is one of those "we don't know yet" moments, so be careful with wildcards on systems that don't support them. 💀
properties is a token list of keywords, and each keyword is the name of a property. 🎀 If properties is left out or given as the empty token list, the server sends along all properties. If any properties are supplied, the user is asking the server to send only those properties.
control-keywords ARGUMENT TO DIRECTORY
control-keywords is a token list of keywords. 🦋 These keywords change how the DIRECTORY command behaves on the server machine. Some of the options below ask the server to filter the data it returns, but it is never an error for the server to return more info than was asked for. 🫶
The following keywords are recognized:
DELETED
This includes soft-deleted files in the directory list. Without this option, they must not be included. 🙅♀️ Those files have the DELETED property set to "true" among their properties. DELETED gets ignored on systems that don't support soft deletion.
DIRECTORIES-ONLY
This option changes the semantics of DIRECTORY pretty drastically. 😭 Normally, the server returns info about all files, directories, and links whose pathnames match the supplied pathname. That means each file, directory, or link that gets listed needs its directory name to match the (potentially wildcarded) directory name in the supplied pathname, its file name needs to match the file name in the supplied pathname, and so on.
When DIRECTORIES-ONLY is supplied, the server is to list only directories: not the ones whose pathnames match the supplied pathname, but the ones whose pathnames, expressed as directory pathnames, match the (potentially wildcarded) directory portion of the supplied pathname. 🧚 The description of the PROBE-DIRECTORY keyword, which can be supplied as the direction argument of the OPEN command, talks about this too. See the section "OPEN Command", section 8.20.
It isn't established yet what servers on hosts that don't natively support this kind of action are supposed to do when they get DIRECTORIES-ONLY along with a pathname that has a wildcard directory component. 🤔
FAST
Speeds up the operation and the data transmission by not listing any properties at all for the files concerned. 🚀 In other words, only the truenames are returned.
NO-EXTRA-INFO
This tells the server to skip listing properties that are generally harder or more expensive to get. 💅 It usually drops the directory-specific stuff, like info about default generation counts and expunge dates.
SORTED
This makes the directory listing get sorted. 🍓 The sorting goes alphabetically by directory, then by file name, then by file type, then by file version (by increasing version number).
8.11.1 NFILE DIRECTORY Data Format
If the NFILE DIRECTORY command completes successfully, a single token list with the requested directory info gets sent on the data channel named by the input-handle argument in the DIRECTORY command. 💖 This section describes the format of that single token list, and it also gives more detail on the properties argument to DIRECTORY.
The token list is a top-level token list, so it's delimited by TOP-LEVEL-LIST-BEGIN and TOP-LEVEL-LIST-END. 👑 The top-level token list holds embedded token lists. The first embedded token list contains the empty token list, followed by property/value pairs that describe the file system as a whole rather than any one file. NFILE requires one file system property to be present: DISK-SPACE-DESCRIPTION, which is a string describing how much free file space the system has. ✨ The embedded token lists that follow each contain the pathname of a file, followed by property/value pairs describing that file.
Here's an example of the top-level token list that DIRECTORY returns, for two files. 🎀 The server is expected to return several property/value pairs for each file, but the number of pairs isn't constrained. In this example, two pairs are returned for the file system, two pairs for the first file, and only one pair for the second file.
TOP-LEVEL-LIST-BEGIN
LIST-BEGIN - first embedded token list starts
LIST-BEGIN - an empty embedded token list starts
LIST-END - the empty embedded token list ends
prop1 value1 - property/value pairs of file system
prop2 value2
LIST-END
LIST-BEGIN
pathname1 - pathname of the first file
prop1 value1 - property/value pairs of first file
prop2 value2
LIST-END
LIST-BEGIN
pathname2 - pathname of the second file
prop1 value1 - property/value pairs of second file
LIST-END
TOP-LEVEL-LIST-ENDThis next example is designed to show the structure of the top-level token list. 🦋 TOP-LEVEL-LIST-BEGIN and TOP-LEVEL-LIST-END are shown as parentheses, and LIST-BEGIN and LIST-END are shown as square brackets, respectively. The indentation, blank spaces, and newlines in the example are not part of the token list. They're just there to make the structure easier to see. 🫶
([ [ ] prop1 value1 prop2 value2]
[pathname1 prop1 value1 prop2 value2]
[pathname2 prop1 value1])The pathname is a string in the full pathname syntax of the server host. 🌈 See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
For more info on file property/value pairs, see the section "Format of NFILE File Property/Value Pairs", section 7.5. 💜
8.12 DISABLE-CAPABILITIES Command 🔒
Command: (DISABLE-CAPABILITIES tid capability)
Response: (DISABLE-CAPABILITIES tid cap-1 success-1
cap-2 success-2 cap-3 success-3 ...)DISABLE-CAPABILITIES turns off an access capability on the server machine. 🚫 capability is a string naming the capability to disable. What the capability actually means depends on the operating system. 🖥️
The return values cap-1, cap-2, and so on, are strings that give the names of capabilities. If the capability named by cap-1 was disabled successfully, the matching success-1 comes back as Boolean truth. Otherwise it's the empty token list. ✨
Even though the user can only specify one capability to disable, disabling it could end up disabling other, related capabilities too. 🌸 That's why the command response can have info about more than one capability. Iconic behavior, honestly. 💅
8.13 ENABLE-CAPABILITIES Command 🔓
Command: (ENABLE-CAPABILITIES tid capability password)}
Response: (ENABLE-CAPABILITIES tid cap-1 success-1
cap-2 success-2 cap-3 success-3 ...)ENABLE-CAPABILITIES turns on an access capability on the server machine. 🦋 The password argument is optional, and it should only be included if this particular capability needs one to be enabled. Both password and capability are strings. The meaning of the capability depends on the operating system. 🖥️
The return values cap-1, cap-2, and so on, are strings that give the names of capabilities. If the capability named by cap-1 was enabled successfully, the matching success-1 comes back as Boolean truth. Otherwise it's the empty token list. 🎀
Same deal as before: the user can only ask for one capability to be enabled, but enabling it might turn on other related capabilities too. 🌈 That's the reason the command response can include info on more than one capability. 💖
8.14 EXPUNGE Command 🗑️
Command: (EXPUNGE tid directory-pathname)
Response: (EXPUNGE tid server-storage-units-freed)EXPUNGE makes the directory named by pathname get expunged. ✨ Expunging means any files that were soft deleted get permanently removed. 💀
For file systems that don't support soft deletion, the command is to be ignored. A success command response still gets sent, but no action happens on the file system. 🙈 In this case, the number-of-server-storage-units-freed return value should be omitted.
directory-pathname is a required string argument in the directory pathname format. 🫶 It has to refer to a directory on the server file system, not to a file. See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
The return value server-storage-units-freed is an integer saying how many records, blocks, or whatever unit is used to measure file storage on the server host system got recovered. 🍓 This return value should be omitted if the server doesn't know how many storage units were freed.
The protocol doesn't define whether directory-pathname is really a pathname for a directory, or a wildcard pathname of files to be expunged. 🤷♀️ It also doesn't define whether wildcards are permitted, or required to be supported, in the directory portion of the pathname (which would be an implicit request to expunge many directories). Main character energy of undefined behavior, not gonna lie. 👑
8.15 FILEPOS Command ✨
Command: (FILEPOS tid handle position resync-uid)
Response: (FILEPOS tid)okay so FILEPOS is the command that moves the file access pointer to a specific spot, counting from the very beginning of the file 🎀 It's how you tell the server where the next byte of data should be transferred. main character energy 💅
The handle says which file gets affected. It has to be a data channel handle for a data stream opening, or a direct file identifier for a direct access opening. Both handle and position are required arguments, no exceptions 🙅♀️
position is an integer for the spot the pointer gets reset to. It's either a byte number based on the current byte size, or a character count for character openings. Zero is the beginning of the file 🌸 If it's a character opening, the position is measured in server units, not NFILE character set units. (the units matter, bestie 🤓)
If FILEPOS is given on an input data channel (aka a data channel currently sending data from server to user), that channel has to be resynchronized after the FILEPOS is done so it can find the start of the new data 🦋 The resync-uid is a unique identifier tied to that resynchronization, and it's unique within this dialogue. resync-uid must be supplied if handle is an input handle, but it's not supplied otherwise. For the full resynchronization procedure, check the section "NFILE Data Connection Resynchronization", section 9.2 ✨
For the output case, the user has to somehow tell the server on the output data channel when there's no more data. They do that by sending the keyword token EOF. Once the server gets that control token, it is required to position the file pointer according to the position given. When the new file position is set, the server picks back up accepting data at that spot 🧚
In most cases, direct access mode is more convenient and efficient than repeatedly using FILEPOS with a data stream opening. just saying 💜
There are some real problems with setting a file position on a character-oriented file on a foreign host when one machine is a Symbolics computer and the other one isn't. For example, character set translation has to happen. See the section "NFILE Character Set", section 6. Because of these difficulties, FILEPOS might not be supported on character files in the future 😭 FILEPOS is not problematic for binary files, though, so we're good there 🫶
8.15.1 Implementation Hint for FILEPOS Command
The server's processing of this command (done by the control connection handler) must not attempt to wait for the resynchronization procedure to complete. Here's the tea: the user could abort between sending the FILEPOS command and reading for the mark and resynchronization identifier. That scenario could leave the sender of the resynchronization identifier, on the server side, blocked for output indefinitely 💀
Only two commands received on the control connection can break the data channel out of that blocked state: CLOSE with abort-p supplied as Boolean truth, and RESYNCHRONIZE-DATA-CHANNEL. So the control connection must not wait for the data channel to finish the resynchronization procedure. That wait should be handled by the process managing the data channel instead ✨
8.16 FINISH Command 🔥
Command: (FINISH tid handle)
Response: (FINISH tid truename binary-p other-properties)FINISH closes a file and then reopens it right away with the file position pointer saved, so it stays open for more I/O 🦋 If possible, the implementation should do the closing and opening as one indivisible operation, so no other process can get access to the file in between. super safe energy
The arguments, results, and their meaning are identical to those of the CLOSE command. See the section "CLOSE Command", section 8.3. FINISH requires a handle, which has the same meaning as the handle of the CLOSE command 💖
For the output case, in both direct mode and data stream mode openings, the server writes out all buffers and sets the byte count of the file. The user sends the keyword token EOF on the data channel to show that the end of data has been reached. The server leaves the file in a state where, if the system or server crashes any time after the FINISH command completes, it would later appear as though the file had been closed by this command. But the file isn't actually closed right now. It's left open for further I/O operations 🎀 FINISH is a reliability feature, and that's iconic behaviour 👑
FINISH is somewhat pointless in the input case, but it's still valid 🙈 The native Symbolics file system (LMFS) implements FINISH on an output file with an internal operation that effectively goes through the work of closing but leaves the file open for appending.
ERRORS ON FINISH
After writing every last bit sent by the user to disk, and before closing the file, the server checks the data channel specified by handle to see if an asynchronous error is outstanding on that channel. In other words, the server must determine whether it has sent an asynchronous error to the user that the user hasn't responded to with a CONTINUE command yet. If so, the server is unable to finish the file, and it must send a command error response indicating that an error is pending on the channel. The appropriate three-letter error code is EPC 😤 See the section "NFILE Errors and Notifications", section 10.
8.17 HOME-DIRECTORY Command 🌈
Command: (HOME-DIRECTORY tid user)
Response: (HOME-DIRECTORY tid directory-pathname)HOME-DIRECTORY returns the full pathname of the home directory on the server machine for the given user 🏡
user is a string that should be recognizable as a user's login name on the server operating system. directory-pathname is a string in the directory pathname format. See the section "Syntax of File and Directory Pathname Arguments", section 7.4 🧚
8.18 LOGIN Command 🔑
Command: (LOGIN tid user password FILE-SYSTEM USER-VERSION)
Response: (LOGIN tid keyword/value-pairs)LOGIN logs the given user in to the server machine, using the password if necessary. Both user and password are string arguments. user is required, and password is optional. An omitted password is valid if the host lets the specified user log in without one 🙌 Depending on the operating system and server, you might need to log in to run a program (in this case, the NFILE server program) on the host. LOGIN establishes a user identity that the operating system uses to set the file author and decide file access rights during the current session 💅
The server has the option to reject with an error any command except LOGIN if a successful LOGIN command hasn't been performed. This is recommended 🫶 Lots of operating systems do the login function in a different process and/or environment than user programs. The part of the NFILE server running in that special login environment could conceivably only be capable of processing the LOGIN command. That's the reason LOGIN exists in NFILE, iykyk 🦋
FILE-SYSTEM and USER-VERSION are optional keyword/value pairs. The FILE-SYSTEM keyword/value pair selects the identity of the file system that all following commands in this session get directed to. This only has meaning if the server host has multiple file systems, and the targeted one is different from the default file system a user gets by starting a dialogue with that host. The FILE-SYSTEM argument is an arbitrary token list. If the server doesn't recognize it, the server gives an appropriate command error response 🙅♀️
Right now, the only use of FILE-SYSTEM is for Symbolics servers to select one of the front-end processor hosts instead of the LMFS, which is the default. In that case, the first element in the token list is the keyword FEP, and the second element is an integer indicating the desired FEP disk unit number. If the server discovers there's no such file system, it gives a command error response including the three-letter code NFS, meaning "no file system". See the section "NFILE Errors and Notifications", section 10 ✨
The user tells the server what version of NFILE it's running by including the optional USER-VERSION keyword/value pair. The value for USER-VERSION can be a string, an integer, or a token list. This document describes NFILE user version 2 and server version 2 💖
When the server gets the user version, it can either adjust certain parameters to handle that particular version, or simply ignore the user version altogether. Currently, the only released versions of NFILE are user version 2 and server version 2 🌸
LOGIN RETURN VALUES
The keyword/value-pairs is a token list made up of keywords followed by their values. The server includes any or all of the following keywords and their values, and they're all optional. These are the recognized keywords 👑
NAME
The value is a string specifying the user identity, in the server host's terms.
PERSONAL-NAME
The value is a string for the user's personal name, last name first. For example: "McGillicuddy, Aloysius X." 🎀
HOMEDIR-PATHNAME
The value is a string in the pathname as directory format, indicating the user's home directory. See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
GROUP-AFFILIATION
The value is a string specifying the group the user belongs to, when that concept is appropriate.
SERVER-VERSION
The value can be a string, an integer, or a token list. It's a representation of the version of the server that is running. When the user receives the server version, they can adjust certain parameters to handle that particular version, accept the version, or close the connection. Currently, the only released versions of NFILE are user version 2 and server version 2 ✨
PROPERTY-INDEX-TABLE
The value is a token list of keywords. This return value lets the server tell the user which file properties are meaningful on its file system. The keywords in PROPERTY-INDEX-TABLE can be used by the DIRECTORY command (a user request for information on file properties of a specified directory or directories). The server can specify a certain property by giving an integer, which is the index of that file property into the PROPERTY-INDEX-TABLE. This cuts down the volume of data sent during directory listings 🧚 The first element in PROPERTY-INDEX-TABLE is indexed by the number 0. See the section "DIRECTORY Command", section 8.11 💜
8.19 MULTIPLE-FILE-PLISTS Command 🦋
Command: (MULTIPLE-FILE-PLISTS tid input-handle paths
characters properties)
Response: (MULTIPLE-FILE-PLISTS tid)okay so MULTIPLE-FILE-PLISTS is the bulk-lookup queen 👑 You hand the server a whole pile of pathnames and it sends back file property info for all of them in one go. The server sends that info as a data structure (the format is explained later in this section) on the given input-handle. 💅
paths is an embedded token list of the pathnames the user is interested in 🍓 Each one is a string in the full pathname syntax of the server host. Unlike for the DIRECTORY command, wildcards are not allowed in these pathnames. See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
characters is either Boolean truth (meaning each file is a character file), the empty token list (meaning each file is a binary file), or the keyword DEFAULT. DEFAULT means the server itself figures out whether a file is character or binary 🧚 For more on what DEFAULT means: see the section "OPEN Command", section 8.20. The value of characters can influence some servers' idea of a file's length, so it's not nothing! 🙈
properties is a token list of keywords saying which properties the user wants back. The server is always free to return more properties than were requested in the properties argument ✨ If properties is supplied as the empty token list, the server should transmit all known properties on the files.
The server sends as much of the requested info as it possibly can on the given input-handle 🫶 The info lives in a top-level token list of elements. Each element lines up with one supplied pathname, and the order of the original pathlist must be retained in the returned token list, so nothing gets mixed up. An element is an empty token list if the corresponding file, or any of its containing directories, does not exist. The elements for successfully located files are lists made of the truename followed by any properties. Properties are keyword/value pairs, and truename is a string in the full pathname syntax of the server host.
The following example shows TOP-LEVEL-LIST-BEGIN and TOP-LEVEL-LIST-END as parentheses, and LIST-BEGIN and LIST-END with square brackets. Bestie, the brackets are doing a lot here 💜
For example, say the user supplied a pathlist argument that looks like this:
[file1 file2 file3]The server couldn't locate file1 or file3, but it did find file2, and it found the length and author of file2. So the top-level token list the server sends back is:
( [] [ truename-of-file2 LENGTH 381 AUTHOR williams ] [] )For more detail on how file properties and values are expressed: see the section "Format of NFILE File Property/Value Pairs", section 7.5. 🌈
8.20 OPEN Command 🎀
Command: (OPEN tid handle pathname direction binary-p
TEMPORARY RAW SUPER-IMAGE DELETED PRESERVE-DATES
SUBMIT DIRECT-FILE-ID ESTIMATED-LENGTH BYTE-SIZE
IF-EXISTS IF-DOES-NOT-EXIST)
Response: (OPEN tid truename binary-p other-properties)OPEN opens a file for reading, writing, or direct access at the server host. In general, that means asking the host file system to access the file and getting back a file number, pointer, or some other quantity for quick access later. This is called an "opening" 🦋 See the section "NFILE File Opening Modes", section 5.
The OPEN command has the most complicated syntax of any NFILE command, no cap 😭 It has required arguments, an optional argument, and a whole bunch of optional keyword/value pairs. For the syntax details of each of these parts: see the section "Conventions Used in This Document", section 7.
The following arguments are required: pathname, direction, and binary-p. handle is an optional argument. It has to be either supplied or explicitly left out by putting the empty token list in its place.
The OPEN command has many optional keyword/value pairs. They encode conceptual arguments for the server file system's OPEN operation. A detailed description of every supported OPEN optional keyword is given below 🧚
The OPEN return values tell you about the file that got opened, as long as the opening succeeds. For a probe-type opening, that info comes back when the given file (or link, or directory) exists and is accessible, even though it isn't actually opened. For details on the OPEN return values: see the section "NFILE OPEN Response Return Values", section 8.20.2.
THE pathname OPEN ARGUMENT
pathname is the required argument that names the file to be opened. It's a string in the full pathname syntax of the server host. See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
For some purposes (for example, when the OPEN argument direction is supplied as PROBE-DIRECTORY), only the directory specified by this pathname gets used. See the section "NFILE OPEN Optional Keyword/Value Pairs", section 8.20.1.
THE handle OPEN ARGUMENT
handle is the argument that picks the data channel used for the transfer. Later commands in this session use the same handle to refer to this opening 💖 It's the user side's job to make sure handle points at an existing, free data channel that doesn't need resynchronization before use. A handle must be supplied, unless you want a probe-type opening (that is, the direction is supplied as PROBE, PROBE-DIRECTORY, or PROBE-LINK) or you're requesting a direct access opening (that is, a DIRECT-FILE-ID is supplied). In those cases, the empty token list goes in for handle.
THE direction OPEN ARGUMENT
direction has to be one of these keywords: INPUT, OUTPUT, IO, PROBE, PROBE-DIRECTORY, and PROBE-LINK. Here's what each one means 🍓
INPUT
Specifies that the file is to be opened for input (server-to-user transfer). To request a direct access opening, supply a value for DIRECT-FILE-ID. If no DIRECT-FILE-ID is supplied, the opening is a data stream opening.
OUTPUT
Specifies that the file is to be opened for output (user-to-server transfer). To request a direct access opening, supply a value for DIRECT-FILE-ID. If no DIRECT-FILE-ID is supplied, the opening is a data stream opening.
IO
Specifies that interspersed input and output will be performed on the file. This is only meaningful in direct access mode, and a DIRECT-FILE-ID must also be supplied. See the section "NFILE OPEN Optional Keyword/Value Pairs", section 8.20.1.
If direction is supplied as PROBE, PROBE-LINK, or PROBE-DIRECTORY, the opening is said to be a probe-type opening. The DIRECT-FILE-ID option is meaningless and an error for probe-type openings. The file handle must be supplied as an empty token list for probe-type openings 🙅
PROBE
Specifies that the file is not to be opened at all, just checked for existence. If the file does not exist or isn't accessible, the error indications and actions are identical to what would happen for an INPUT opening. If the file does exist, the successful command response contains the same info it would have had if the file had been opened for INPUT. If it's a link, the link is followed to its target.
PROBE-LINK
Like PROBE, with one difference. PROBE-LINK specifies that if the pathname turns out to refer to a link, that link is not to be followed, and information about the link itself is returned instead 🔗
PROBE-DIRECTORY
PROBE-DIRECTORY requests information about the directory designated by the pathname argument. In the PROBE-DIRECTORY case, the pathname argument refers to the directory the info is requested about. In all other cases, the pathname refers to a file to be opened. If pathname contains a file name and file type, those parts are ignored for PROBE-DIRECTORY openings, as long as they're syntactically valid 🌸
THE binary-p OPEN ARGUMENT
The value of binary-p affects the mode the server opens the file in, and it tells the server whether character set translation has to happen ✨
If binary-p is supplied as the empty token list, the opening is said to be a character opening. The server translates characters between its native character set and the NFILE character set. The data goes over the data connection one character per eight-bit byte. See the section "NFILE Character Set", section 6.
If binary-p is supplied as Boolean truth, the opening is said to be a binary opening. The user side supplies the byte size via the BYTE-SIZE option. If it isn't supplied, the default byte size is 16 bits. If the byte size is less than 9, the file data is transferred byte by byte. If the byte size is 9 or greater, the server transfers each byte of the file as two eight-bit bytes, low-order first 🌈
binary-p can also be supplied as the keyword DEFAULT. DEFAULT means the server itself figures out whether to transfer binary or character data. DEFAULT is only meaningful for input openings. It's an error for OUTPUT, IO, or probe-type openings. For file systems that keep track of whether a file is innately binary or character, the server simply asks the file system which one is in force for the file named by pathname.
When binary-p is supplied as DEFAULT on file systems that don't keep track of this information, the server is required to do a heuristic check for Symbolics object files, looking at the first two 16-bit bytes of the file. If the file is determined to be a Symbolics object file, the server performs a BINARY opening with BYTE-SIZE of 16. Otherwise, it performs a CHARACTER opening 👀
The details of the check go like this: if the first 16-bit byte is the octal number 170023, and the second 16-bit byte is any number between 0 and 77 octal (inclusive), the file is recognized as a Symbolics object file. In any other case, it is not.
8.20.1 NFILE OPEN Optional Keyword/Value Pairs 🍓
The OPEN command has many optional keyword/value pairs that encode conceptual arguments for the file system's OPEN operation 💅
The following option is used often:
BYTE-SIZE
Here the rule is that it must be followed by an integer between 1 and 16, inclusive, or by the empty token list. BYTE-SIZE is meaningful only for binary openings. It can be ignored for probe-type openings. It can be left out entirely for character openings, but if it is supplied, it must be followed by the empty token list. If binary-p is supplied as DEFAULT, BYTE-SIZE can be left out entirely, or followed by the empty token list.
If a binary opening is requested and BYTE-SIZE is not supplied, the assumed value is 16 for output openings. For input binary openings, the default is the host file system's stored conception of the file's byte size (for those hosts that natively support byte size). For file systems that do not natively support byte size, the default byte-size on binary input is 16 🦋
Okay so for file systems that keep track of each file's innate byte size 🧚, the server should hand that number to the operating system interface that performs the open. Other operating systems are a little more relaxed, but there's one big rule: a file written with a given byte size must produce the same bytes in the same order when it's read back with that byte size 💖. As long as that holds, the server or host OS can pick any packing scheme that follows the rule. Let it cook 🍓
Operating systems that don't support byte size must ensure that binary files written from the user ends of the current protocol can be read back correctly ✨. The server can also choose packing schemes that let every bit of the server host's word be accessed, as long as they agree with the other packing schemes that native host software uses 🦋. All the bits, all the time 💅
Multics supports 36-bit words and 9-bit bytes, so it's the ultimate word glow-up 👑. A packing scheme that works for a Multics NFILE server looks like this:
Byte Size Packing Scheme
7, 8, or 9 bits four per 36-bit word
10, 11, or 12 bits three per 36-bit word
13, 14, 15, or 16 bits two per 36-bit wordIn the first packing scheme in the table, native Multics character-oriented software can access each logical byte one at a time, in order 🌸. In the last one, each Symbolics byte lives in a halfword, so it's easy to reach and easy to see in an octal representation 👀. For maximum data transfer rate and access to all the bits of a Multics word, a byte size of 12 must be specified 🎀
DELETED
If supplied as Boolean truth, DELETED means that deleted files are to be treated as though they were not "deleted" 🙈. It only matters for operating systems that support "soft deletion" and later "undeletion" of files. Everyone else must ignore this option. Normally, deleted files are invisible to the OPEN operation, and this option makes them visible 👻
DELETED can also be followed by the empty token list, which does the same thing as leaving the keyword/value pair out entirely. For output openings, DELETED is meaningless, and supplying it is an error 💀
DIRECT-FILE-ID
If supplied, DIRECT-FILE-ID means the opening is a direct access mode opening 🎯. If it's not supplied, the opening is a data stream opening. The value of DIRECT-FILE-ID is a string that the user generates. It can't have been used as a DIRECT-FILE-ID earlier in this dialogue, and it can't designate any data channel. It's a unique identifier for the direct access opening and gets used for every operation that identifies an opening rather than a data channel 💜. A direct access opening uses DIRECT-FILE-ID the way a data stream opening uses a file handle. The PROPERTIES, CLOSE, and RENAME commands all use DIRECT-FILE-ID like this. There are only two NFILE commands that apply to direct access openings and don't use DIRECT-FILE-ID: ABORT and CONTINUE. They use a data channel handle instead 🔥
PRESERVE-DATES
If supplied as Boolean truth, PRESERVE-DATES tells the server to try to stop the operating system from updating the "reference date" (the date-time used) of the file 📅. This only matters for input openings, and using it anywhere else is an error.
The Symbolics operating system turns this on for things like View File in the editor, where it's basically saying "I didn't read this, I just looked at it" 👀. Servers on operating systems that don't support reference dates, or that don't let users revise or suppress updates to them, must ignore this option.
ESTIMATED-LENGTH
The value of ESTIMATED-LENGTH is an integer that estimates how long the file to be transferred is 📏. This only matters and is only allowed for output openings. It lets the user end give the server's file system a heads-up on how big the file is going to be, which is super helpful for file systems that have to preallocate files or file maps, or that get performance perks from knowing this when the file first gets opened 🚀. The estimate doesn't have to be exact if it's supplied. Servers that don't find it useful or interesting just ignore it.
The units of the estimate are characters for character openings, and bytes of the agreed-upon byte size for binary openings. Character units should be server units if possible, but since it's only an estimate, NFILE character units are fine too. See the section "NFILE Character Set", section 6 📚
IF-EXISTS
This one only matters for output openings, and it's ignored otherwise without being flagged as an error 🤷♀️. The value of IF-EXISTS is a keyword that says what to do if a file with the given name already exists. The meanings come from the Common Lisp specification and are repeated here for completeness 💅. If the file doesn't already exist, the IF-EXISTS option and its value are ignored.
If the user side doesn't give the IF-EXISTS option, the default depends on whether the file system supports file versions. If it does, the default is ERROR (if an explicit version is given in the file pathname) or NEW-VERSION (if the version in the file pathname is the newest version). For file systems that don't support versions, the default is SUPERSEDE. These actions are described below.
IF-EXISTS is the thing that handles overwriting or appending to files. With the default setting, every output opening creates a new file 🌈
Operating systems that support soft deletion can do something different when a "deleted" file already exists with the same name (and type and version, where that applies) as a file about to be created ✨. The Symbolics file system (LMFS) basically uses SUPERSEDE even when nobody asked it to. Other servers and file systems are urged to do the same. The recommended move is to not let deleted files stop a file from being created successfully (with a specific version number), even if an IF-EXISTS option weaker than SUPERSEDE, RENAME, or RENAME-AND-DELETE is specified or implied 🙅♀️
Here are the possible values and what they mean:
ERROR: Reports an error.NEW-VERSION: Creates a new file with the same file name but with a larger version number. This is the default when the version component of the filename is newest. File systems without version numbers can implement this by effectively treating it asSUPERSEDE.RENAME: Renames the existing file to some other name and then creates a new file with the specified name. On most file systems, this renaming happens at the time of a successful close.RENAME-AND-DELETE: Renames the existing file to some other name and then deletes it (but doesn't expunge it, on systems that tell deletion and expunging apart). Then it creates a new file with the specified name. On most file systems, this renaming happens at the time of a successful close.OVERWRITE: Output operations on the opening destructively modify the existing file. New data replaces old data at the beginning of the file, but the file isn't truncated to length zero when it's opened 🦋TRUNCATE: Output operations on the opening destructively modify the existing file. The file pointer starts at the beginning of the file. At that point,TRUNCATEcuts the file down to length zero and frees the disk storage it was using 🧹APPEND: Output operations on the opening destructively modify the existing file. New data goes at the current end of the file 📝SUPERSEDE: Supersedes the existing file. That means the old file is removed or deleted and expunged, and the new file takes its place. If possible, the file system doesn't destroy the old file until the new file is closed, in case the file gets close-aborted. This is different fromNEW-VERSIONbecauseSUPERSEDEcreates a new file with the same name as the old one, instead of a file name with a higher version number 💖
There are currently no standards on what a server can do if it can't implement some of these actions 🤔
IF-DOES-NOT-EXIST
This one is meaningful for input openings, never meaningful for probe-type openings, and sometimes meaningful for output openings 🌟. IF-DOES-NOT-EXIST takes a value token that says what to do if the file doesn't already exist. Like IF-EXISTS, it's borrowed from Common Lisp. The default goes like this: if this is a probe-type opening or a read opening, or if IF-EXISTS is specified as OVERWRITE, TRUNCATE, or APPEND, the default is ERROR. Otherwise, the default is CREATE.
These are the values for IF-DOES-NOT-EXIST:
ERROR: Reports an error.CREATE: Creates an empty file with the specified name and then proceeds as if it already existed 🌱
The following optional keyword/value pairs are rarely used, if ever 🕰️
RAW
If supplied as Boolean truth, RAW says that character set translation is not to be performed, and characters should be transferred intact without being inspected 🧼. This is only meaningful for character openings, and it's an error otherwise. It's also an error to supply RAW as Boolean truth for probe-type openings. RAW can also be followed by the empty token list, which does the same thing as leaving the RAW keyword/value pair out entirely. See the section "RAW Translation Mode", Appendix B.
SUPER-IMAGE
If supplied as Boolean truth, SUPER-IMAGE says that Rubout quoting is not to be performed 🎀. This operation only means something for character openings, and it's an error otherwise. It's also an error for probe-type openings. SUPER-IMAGE can also be followed by the empty token list, which does the same thing as leaving the SUPER-IMAGE keyword/value pair out entirely.
In SUPER-IMAGE mode, the server reads or writes character files where ASCII Rubout characters are a real part of the file content, not an escape for this protocol 🤯 However, other translations still have to happen. See the section "SUPER-IMAGE Translation Mode", Appendix C.
TEMPORARY
This one is used by the TOPS-20 server only 🖥️. TEMPORARY says to use GJ%TMP in the GTJFN. It's mostly useful when writing files, and it tells the foreign operating system to treat the file as temporary. See the TOPS-20 documentation for more about what this option implies. Other servers can ignore it. It's meaningless and an error for input or probe-type openings. TEMPORARY can also be followed by the empty token list, which does the same thing as leaving the TEMPORARY keyword/value pair out entirely 🧚
SUBMIT
The SUBMIT details pick up right where this part leaves off 💅🎀
SUBMIT is meaningful for output only 💅 If it's supplied as Boolean truth, SUBMIT tells the server to hand the contents of the file being written to the operating system as a job, once the file is closed ✨ VMS is an example of an operating system that could conveniently support SUBMIT 🌸 SUBMIT can also be followed by the empty token list, which does exactly the same thing as if the SUBMIT keyword/value pair were left out entirely 🫶 Servers that do not implement this option should give an error response if someone asks them to submit a file to the operating system 🙈
8.20.2. NFILE OPEN Response Return Values 🦋
Okay so the results of a successful OPEN show up in the command response. Here's the response format:
Response Format:
(OPEN tid truename binary-p other-properties)The return values for OPEN and CLOSE are syntactically identical, but the values can change in the time between open and close 👑
truename is a string with the pathname of the file in the full pathname syntax of the server host. The server should figure it out once it has opened the file, by asking its operating system something like "What file corresponds to this JFN, file number, pointer, etc.?" 💖 If the operating system supports version numbers, this string always has an explicit version number. It always has a directory name, a file name, and so on 🧚
Some operating systems might not know the truename of an output file until it's closed. It's permissible not to supply an explicit version number in the pathname in the OPEN response in this specific case. On those systems, the truename when the file is opened is different from the truename after it's closed 🌈
The return value binary-p tells you whether the opening is a binary or character opening. For binary openings, binary-p is supplied as Boolean truth. For character openings, it's the empty token list 🎀
other-properties is a list of keyword/value pairs. other-properties must contain CREATION-DATE and LENGTH. AUTHOR should be included if the server operating system has a convenient mechanism for determining the author of the file. The other properties described here can be included if you want 💅
AUTHOR
The value of AUTHOR is a string with the name of the author of the file. This is some kind of user identifier, and its format is system-specific. Like CREATION-DATE (see below), AUTHOR is supposed to represent the logical determinor of the current data content of the file, not necessarily the agency that actually created the file 🫶
BYTE-SIZE
The byte-size is agreed upon via the rules described for the BYTE-SIZE option. The value is an integer. For the details on what BYTE-SIZE means, see the section "NFILE OPEN Optional Keyword/Value Pairs", section 8.20.1. This parameter is only meaningful for BINARY openings. However, if FILEPOS is returned in the other-properties list, BYTE-SIZE should also be included, even for character openings ✨
CREATION-DATE
The creation date of the file. The date is expressed in Universal Time format, which measures a time as the number of seconds since January 1, 1900, at midnight GMT. Creation date doesn't necessarily mean the time the file system created the directory entry or records of the file. For systems that support modification or appending to files, it's usually the modification date of the file. Creation date can also mean the date that the bit count or byte count of the file was set by an application program 🕰️
Some types of file systems support a user-settable quantity (CREATION-DATE) which the user can set to an arbitrary time, to indicate that the contents of this file were written a long time ago by someone else on another computer. The default value of this quantity, if the user hasn't set it, is the time someone last modified the information in the file. This quantity, in the OPEN response for an output file, is disregarded by the user side, but nevertheless must be present 💜
The Symbolics computer system software uses this quantity as a unique identifier of file contents, for a given file name, type, and version, to prove that a file has not changed since it last recorded this quantity for a file 🔥
FILEPOS
An integer giving the position of the logical file pointer, in characters or bytes as appropriate for the type of opening. It's always zero for an input opening and for an output opening creating a new file. For an output opening appending to an existing file, FILEPOS is the number of characters or bytes, as appropriate, currently in the file. For character openings, this number is measured in server units: see the section "NFILE Character Set", section 6 🧚
LENGTH
An integer reporting the length of the file, in characters for character openings and in bytes of the agreed-upon size for binary openings. LENGTH should be reported as zero for output openings, even if appending to an existing file. The server usually only knows the length for a character opening in server units, so it reports length in server units 🌸
8.21. PROPERTIES Command
Command: (PROPERTIES tid handle pathname control-keywords
properties)
Response: (PROPERTIES tid property-element settable-properties)PROPERTIES requests the property information about one file 🦋 The file is identified by the pathname argument or the handle argument, but not both. If pathname is supplied, it's a string in the full pathname syntax of the server host. See the section "Syntax of File and Directory Pathname Arguments", section 7.4.
If handle is supplied, its value is a string identifying an opening, which implicitly identifies a file. For direct access mode openings, handle must be a direct file identifier.
control-keywords is reserved in the current design. However, it's a required argument, and must be supplied as the empty token list. Its presence in the NFILE specification allows for future expansion. In the future, the value of control-keywords might affect the listing mode.
properties is a token list of keywords indicating the properties the user wants returned. (In command arguments, properties cannot be specified with integers, such as indices into the Property Index Table.) For a list of keywords associated with file properties, see the section "Format of NFILE File Property/Value Pairs", section 7.5.
The server is always free to return more properties than those requested in the properties argument. If properties is supplied as the empty token list, the server transmits all known properties of the file 💖
PROPERTIES COMMAND RESPONSE
The server returns the property information for the given file in the command response. The PROPERTIES command does not use any data channels. If the specified file does not exist or is not accessible, the server signals an error and includes an appropriate three-letter error code in the command error response. See the section "NFILE Errors and Notifications", section 10 🙈
The return value property-element is a token list. The first element in that token list is the pathname of the file, in the full pathname syntax of the server host. The following elements of the property-element token list are property/value pairs. The server is expected to return several property/value pairs, and the number of pairs is not constrained. For further details on file properties and their associated values, see the section "Format of NFILE File Property/Value Pairs", section 7.5.
The return value settable-properties is a token list of keywords. The number of keywords is not constrained. (Note that integers cannot be used in settable-properties to indicate the file property; keywords are to be used instead.) Each keyword supplied in settable-properties identifies a property considered settable by the server. The server is implicitly guaranteeing a mechanism for changing the properties reported as settable. The user can change any of the settable properties for this file by using the CHANGE-PROPERTIES command. See the section "CHANGE-PROPERTIES Command", section 8.2 🎀
The following example shows the format of the PROPERTIES command response. Remember that the number of property/value pairs and keywords is not constrained. This example has two property/value pairs and three settable-properties keywords returned:
TOP-LEVEL-LIST-BEGIN
PROPERTIES - name of the command
tid - transaction identifier
LIST-BEGIN
pathname of file
prop1 value1 - file's property/value pairs
prop2 value2
LIST-END
LIST-BEGIN
keyword-1 - file's settable properties
keyword-2
keyword-3
LIST-END
TOP-LEVEL-LIST-ENDThe following example is designed to better show the structure of the top-level token list, by depicting TOP-LEVEL-LIST-BEGIN and TOP-LEVEL-LIST-END by parentheses and LIST-BEGIN and LIST-END by square brackets. The indentation and newlines in the example are not part of the token list, but are used here to make the structure of the token list clear 🧚
(PROPERTIES tid [ pathname prop1 value1 prop2 value2 ...]
[ keyword1 keyword2 keyword3 ... ]8.22. READ Command
Command: (READ tid direct-file-id input-handle count FILEPOS)
Response: (READ tid)READ requests input data flow for direct access openings. The direct-file-id is the same as the DIRECT-FILE-ID argument that was given when opening the file. It designates the opening from which the characters or bytes are to be transferred. The input-handle specifies which data channel should be used for the transfer of data from server to user. The data channel should have been already established, cannot have been disestablished, and must not currently be in use 🦋
count is an integer specifying how many bytes (or NFILE characters, as appropriate) to read. count can be supplied as the empty token list, meaning read to the end of the file. If the user specifies the empty token list, or a count greater than the number of bytes remaining in the file, the server sends the keyword EOF to mark the end of the file 💅
FILEPOS is an optional keyword/value pair. If the keyword FILEPOS is supplied, it must be followed by an integer. Before data is transferred, the opening is positioned to the point specified by the value of FILEPOS. The position of the point is measured in server units for character openings. For binary openings, it is measured in binary bytes. See the section "FILEPOS NFILE Command" 🌈
Upon receiving the READ command, the server binds the data channel to the opening and immediately begins transferring data. The server stops when all data has been transferred. After the server sends the last requested byte, it unbinds the data channel, freeing it for other use. When the user side has processed the last byte, the user side assumes that the data channel can now be reused for another data transfer 👑
8.23. RENAME Command 💅✨
Command: (RENAME tid handle pathname to-pathname)
Response: (RENAME tid from-pathname to-pathname)okay so RENAME is how you ask the server to give a file a brand new name 🦋. It's NFILE hitting up the system's native rename operation, so all the system-specific semantics and constraints come along for the ride 💅.
You point at the file with either a handle or a pathname, but not both 🙅♀️. to-pathname is the new name you want. The return values tell you the full original name in from-pathname and the full new name in to-pathname 🌸. Systems that do version numbers might hand back a different version number than the one you sent in, so don't be shook if that happens 😭.
The pathname and to-pathname arguments, and the from-pathname and to-pathname return values, are all strings in the full pathname syntax of the server host. For the details, check the section "Syntax of File and Directory Pathname Arguments", section 7.4 📖.
If the file comes in by pathname, it should be renamed right away. If it comes in by handle, it's totally fine to wait until close-time to do the rename 🫶.
Some operating systems can only rename within a directory, which is a whole mood 🌧️. Even so, to-pathname still has to be fully specified, and on those systems the server must check for and reject a cross-directory rename attempt 🚫.
8.24. RESYNCHRONIZE-DATA-CHANNEL Command 🧹
This one's a shapeshifter 🦎. The command and response format changes depending on whether the handle argument points at an input or an output data channel.
For an Input Handle:
Command: (RESYNCHRONIZE-DATA-CHANNEL tid handle)
Response: (RESYNCHRONIZE-DATA-CHANNEL tid identifier)For an Output Handle:
Command: (RESYNCHRONIZE-DATA-CHANNEL tid handle identifier)
Response: (RESYNCHRONIZE-DATA-CHANNEL tid)RESYNCHRONIZE-DATA-CHANNEL kicks off a prescribed procedure between user and server over the unsafe data channel named by handle ✨. The resync procedure clears the channel of any unwanted data and restores it to a safe state, so it's ready to transfer data again 🧼💖.
All arguments to RESYNCHRONIZE-DATA-CHANNEL are required, no exceptions 🙅♀️.
For a detailed rundown of how the user and server coordinate resynchronizing data channels, check the section "NFILE Data Connection Resynchronization", section 9.2 🎀.
8.24.1. Implementation Hints for RESYNCHRONIZE-DATA-CHANNEL Command 🌷
In general, both the user and server should be implemented knowing that a transmission can be aborted 💅. Basically, the receiving side has to be careful not to act on a transmission (aka perform any action or side effect) until it's been successfully received in its entirety 🫧. That protects the user program from an abort that happens after a transmission has already been partially sent 🛡️.
RESYNCHRONIZING AN OUTPUT DATA CHANNEL 🌊
The server will probably want to hand the looping and reading off to the logical data process 🧚. Looping on the resynchronization identifier inside the control connection handler is not a viable option, bestie 🙈. If the user side never sends the resynchronization identifier (say, because of a user abort), the control connection handler can never break out of that loop 💀.
So the big question: should the user side send the control connection handler command first, or send the marks and identifiers first? 🤔
Sending the marks first is problematic 😬. The data channel on the other end might not be reading them, since it hasn't been told to yet by the control connection handler. The user could then get blocked for output, which would stop them from sending the RESYNCHRONIZE-DATA-CHANNEL command at all 😭.
Sending the control connection handler command first has its own catch 🧩. The user side then needs to be able to send the marks and identifiers between sending that command and receiving its response. The response will never come until the marks and identifiers have been successfully received. So the user implementation has to allow for this one command where a "sends a command and waits for a response" subroutine just doesn't work 🙃.
RESYNCHRONIZING AN INPUT DATA CHANNEL 🌈
The server control process should dispatch the data process to send the mark, and not wait around 🏃♀️. The reason: the data process could get blocked for output because of a user abort. The control process has to go back to its command loop, so it can receive a command that might break the data process out of that block 🔓.
8.25. UNDATA-CONNECTION Command 🔌
Command: (UNDATA-CONNECTION tid input-handle output-handle)
Response: (UNDATA-CONNECTION tid)UNDATA-CONNECTION explicitly disestablishes a data connection from the user side 👋. The user side gets to decide when to disestablish data connections, totally at its discretion 💁♀️. There's no spot in the protocol where disestablishing a data connection is required, except at the end of the session, where it happens implicitly 🌙.
The data connection being disestablished is the one designated by the input-handle and output-handle arguments. Those two handles have to refer to the same data connection 🤝.
You're not allowed to explicitly disestablish a data connection if either of its channels is active 🚫. If the session ends because the control connection breaks, all file handles become meaningless 😵. In that case the server must close all data connections it knows about, and close-abort all the files it opened on behalf of the user during the dialogue 🧹.
On the Symbolics implementation, the user side disestablishes data connections that haven't been used in a long time, like twenty minutes or so ⏰.
For more info about data connections, check the section "NFILE Control and Data Connections", section 4 📚.
9. NFILE RESYNCHRONIZATION PROCEDURE 🔁
Normally, the user side sends NFILE commands to the server side over the control connection 📨. The server side responds to every user command, and the file data goes over the data channels 🚚. This section describes the resynchronization procedure that kicks in when something disturbs the usual flow of events 🌪️.
First up, if the server side aborts while it's sending or receiving data, there's nothing that can be done to salvage the connection between the two hosts 💔. The control connection and any data channels tied to it get broken. Thankfully this happens rarely, if ever 🙏.
The user side aborting file operations (either commands or data transfers) is not unusual at all 😅. On a Symbolics computer, you can do this by pressing CONTROL-ABORT. Honestly, the way a file protocol handles a user-side abort is a huge deal, so this matters a lot 👑.
An NFILE user side reacts to a user-side abort by immediately marking the connection unsafe 🚨. When a control connection is unsafe, it must be resynchronized before it can be used again. Data channels can also be marked unsafe, and they must be resynchronized before any further use too. The resynchronization process clears the connection (whether control or data) of bytes that are now unwanted, so the channel is clean and usable again ✨.
It's a bit complicated, but it fills a genuine need 💅. If you're curious, there's a brief design discussion in note <3> 🧠.
9.1. NFILE Control Connection Resynchronization 🎀
NFILE requires any unsafe control connection to go through a resynchronization procedure before it's used again 🔒. So the resync doesn't necessarily happen right after the control connection gets marked unsafe. The user side starts the control connection resync when another operation on that connection is attempted 🙌.
A "mark" is defined in the context of Byte Stream with Mark. See the section "Discussion of Byte Stream with Mark", section 12.1 🔖.
USER SIDE STEPS: CONTROL CONNECTION RESYNCHRONIZATION 👩💻
- The user side sends a mark over the control connection to the server.
- The user side sends the ASCII characters
USER-RESYNC-DUMMY(as a data token) to the server. - The user side sends a second mark to the server.
- The user side declares the control connection safe (at the token list level).
- The user side generates and sends a unique data token to the server.
- The user side then waits, expecting to detect a mark followed by the unique data token. The user side reads and discards all tokens and marks until the desired match is found.
Once the user side detects the mark and the unique data token, the control connection is fully resynchronized and ready to use again 🥳.
SERVER SIDE STEPS: CONTROL CONNECTION RESYNCHRONIZATION 🖥️
- The server side detects a mark. That alerts the server that the control connection is unsafe and resynchronization is in progress 🚦.
- The server keeps reading data coming from the user side until it detects the second mark and the token following it 👀.
- The server checks whether the token following the mark is
USER-RESYNC-DUMMY. This rare situation happens if the user aborts during the resynchronization itself 😵💫. If so, the server side discards theUSER-RESYNC-DUMMYtoken. The control connection is still unsafe, so the user side restarts the resynchronization procedure, and the server side begins again at Step 2 🔄. - If the token following the mark is not
USER-RESYNC-DUMMY(the expected situation), the server should have received a single data token, which is the unique data token generated by the user side 🎯.- a. The server sends a mark to the user side.
- b. The server declares the control connection safe (at the token list level).
- c. The server sends the unique data token to the user side.
- If the server detects something following the mark that is neither
USER-RESYNC-DUMMYnor a single data token, a protocol error has occurred 🚨💀.
9.2. NFILE Data Connection Resynchronization 🎀
okay so the data channel resync is basically the control connection one's cousin 🦋 Both use the same vibe: a mark to flag the unsafe situation, then a second mark followed by a unique identifier. The big difference is when it happens. Control connections only get messy when the user aborts during control connection I/O. Data channels get messed up by a whole lot more stuff 💀
- User aborts occur during the file protocol operations that assign and deassign data channels. This is the most common cause of data channels becoming unsafe.
- A server receives a CLOSE command (with abort-p supplied as Boolean truth) specifying an open file that has not finished transmitting data. That is, file reading is aborted.
- The ABORT command is issued, causing data channels to be made unsafe.
- The FILEPOS command is issued, causing the input data channel to become unsafe.
The resynchronization clears the data channel of unwanted data from aborted operations and puts the data channel in a known state ✨ It gets kicked off when the user side gives the RESYNCHRONIZE-DATA-CHANNEL command over the control connection.
The following policies can be used to improve response time, but are not required by the NFILE protocol 🌸 The user side can initiate resynchronization only if it needs the data channel, having first tried to use a free data channel that does not require resynchronization. Also, the user side can periodically resynchronize all unsafe data channels. Smart move, bestie 💅
In giving the RESYNCHRONIZE-DATA-CHANNEL command, the user side says which data channel should be resynchronized. Data channels are unidirectional, which means that depending on the direction (input or output) of the data channel, either the user side or the server side sends the resynchronization data. This is another difference from the control connection, where the user side always sends the resynchronization data. The steps for input data channels are different from the steps for output data channels, so no mixing them up 🙈
INPUT DATA CHANNEL RESYNCHRONIZATION
- The user side gives the RESYNCHRONIZE-DATA-CHANNEL command on the control connection, with only one argument, the handle of the data channel to be resynchronized.
- The server side of the data channel generates a unique identifier, and sends that data token in its regular command response to the user side.
- The server side sends a mark over the data channel.
- The server side sends the unique identifier token over the data channel.
- The user side reads until it detects a mark followed by the unique identifier token. The resynchronization is then complete. The data channel is no longer in an unsafe state 🎉
OUTPUT DATA CHANNEL RESYNCHRONIZATION
- The user side gives the RESYNCHRONIZE-DATA-CHANNEL command on the control connection, with two arguments: the handle of the data channel to be resynchronized, and a unique identifier that it has just generated.
- The user side of the data channel sends a mark.
- The user side of the data channel sends a dummy identifier token. The dummy identifier can be any token that the server could not interpret as being the unique identifier. One suggestion is the data token
DUMMY-IDENTIFIER🧚 - The server side of the data channel was alerted by the RESYNCHRONIZE-DATA-CHANNEL command that resynchronization is in progress. The server side now reads the data, seeking the first mark.
- The server side reads and discards the first mark and the dummy identifier.
- The user side sends a second mark.
- The user side sends the unique identifier.
- The server side recognizes the mark and the unique identifier that follows, and the resynchronization is complete. The data channel is no longer in the unsafe state 🦋
10. NFILE ERRORS AND NOTIFICATIONS ✨
okay now for the drama 😭 NFILE recognizes two types of errors: command response errors and asynchronous errors. On top of errors, NFILE also supports notifications, which are the friendly heads-ups 💜
Command response errors:
- Signify an error that prevented the successful completion of the command; when such an error occurs, a command response error is sent instead of a normal command response.
- Occur frequently in normal operations
Asynchronous errors:
- Are not related to any specific command
- Are associated with an erring data channel
- Typically indicate a problem in the transfer, such as running out of disk space or allocation, or an unreadable disk record
- Occur rarely in normal operations
Notifications:
- Are not associated with an error
- Are sent at the server's discretion
- Provide general information, such as a warning that the system is going down
10.1. Notifications From the NFILE Server 📣
The NFILE server can send asynchronous notifications to the user side over the control connection. The text of the notification holds info of interest to the person using NFILE, like a warning that the server's operating system will be going down soon 😱 Notifications can come from the server side at any time that the server is not sending something else.
The format of NFILE notifications is:
(NOTIFICATION "" text)The empty string "" takes the place of a transaction identifier. Notifications are initiated by the server, and are not associated with any transaction originated by the user side. They just pop up, no invite needed 🌈
10.2. NFILE Command Response Errors 🙈
When an error prevents the successful completion of an NFILE command, a command response error is sent instead of the normal command response. A normal command response means success; a command response error means the command failed 💀
NFILE command response errors are sent from the server to the user across the control connection as top-level token lists, in this format:
(ERROR tid three-letter-code error-vars message)ERROR is a keyword. The tid is the transaction identifier of the command that hit this error. The arguments three-letter-code, error-vars, and message are all required.
The three-letter-code tells you what kind of error it was. For a table of the three-letter codes and what they mean, check out the section "NFILE Three-letter Error Codes", section 10.4 🔥
message is a string that gets displayed to the human user of the protocol.
error-vars is a keyword/value list. The three possible keywords are PATHNAME, OPERATION, and NEW-PATHNAME. Before sending an error, the server looks at the type of error to see if it can easily figure out the value of any of the keywords. If so, the server includes that keyword/value pair in its error. If not, the keyword/value pair gets left out. The value tied to OPERATION is the keyword naming the NFILE command that failed. The values tied to PATHNAME and NEW-PATHNAME are strings in the full pathname syntax of the server host.
For example, say the server is on a file system with hierarchical directories and can't access a file because its containing directory doesn't exist. The command error response would use the PATHNAME keyword to point at the first directory level that didn't exist, instead of the full pathname that was supplied as the command argument. That gives the user side valuable info it otherwise wouldn't have known 🧠✨
10.3. NFILE Asynchronous Errors ⚡
When a data channel process, in either direction, runs into an error condition, the server sends an asynchronous error description. An asynchronous error description is a top-level token list. Usually, asynchronous errors point at error conditions in the transfer, like running out of disk space or allocation, or an unreadable disk record 📀
The format of asynchronous error descriptions is:
(ASYNC-ERROR handle three-letter-code error-vars message)ASYNC-ERROR is a keyword. The handle argument identifies the erring data channel. The arguments three-letter-code, error-vars, and message are all required. Their meanings are the same as in NFILE command error responses: see the section "NFILE Command Response Errors", section 10.2.
When the server detects an asynchronous error on an input data channel, it sends the asynchronous error description on that data channel itself. When an asynchronous error occurs on an output data channel, the asynchronous error description is sent on the control connection instead.
Some asynchronous errors are restartable 🔄 In this context, restartable means it makes sense to try to resume the operation. One example is trying to write a file to a file system that's out of room. The server side says whether an asynchronous error is restartable by prepending the keyword RESTARTABLE and its associated value Boolean truth to the error-vars list. To pick up from a restartable error, the user side sends a CONTINUE command over the control connection.
On any asynchronous error, input or output, the data channel on the server side enters an "asynchronous error outstanding" state. The server can leave that state in one of two ways: by receiving a CONTINUE command, or a CLOSE command with the abort-p argument supplied as Boolean truth.
On a normal CLOSE (not a close-abort), the server side checks the channel it was asked to close. If an asynchronous error description has been sent on the data channel but hasn't been processed by CONTINUE yet, the server side does not close the channel, and sends a command error response instead. The same thing happens on a FINISH command received on a channel that has an asynchronous error pending. In both cases, the three-letter code in the command error response is EPC, for Error Pending on Channel 🚫💖
10.4. NFILE Three-letter Error Codes ✨
okay so when something goes wrong, the server's operating system usually has a description of what happened 💅 NFILE has a way to pass that info along to the user side too. When the NFILE server detects an error, it should pick the three-letter code that best describes it 🫶 That code is an argument in both the command response error and the asynchronous error messages the server sends to the user.
Each NFILE three-letter code stands for some system error. Because the set is shared, every operating system can use the same error-reporting mechanism ✨ Some operating systems will never run into certain of these conditions 💀
Some errors could fit two codes. Say the server couldn't delete a file because it wasn't found. That could be CDF (Cannot Delete File) or FNF (File Not Found). FNF is more specific and more useful, so that's the better pick 🙈 The protocol only lets one error code be reported per error, so the server must choose the most appropriate code based on what the operating system told it.
Here's the full set of three-letter codes 🌈
ACC: Access error. This one means there was a protection-violation error.ATD: Incorrect access to directory. A directory couldn't be accessed because the user's access rights didn't allow that type of access 📁ATF: Incorrect access to file. A file couldn't be accessed because the user's access rights didn't allow that type of access.BUG: File system bug. This covers all protocol violations the server detects, plus the ones the host file system detects 🐛CCD: Cannot create directory. Something went wrong while trying to create a directory.CDF: Cannot delete file. The file system said it can't delete a file.CCL: Cannot create link. Something went wrong while trying to create a link.CIR: Circular link. An operation was attempted on a pathname that designates a link which eventually links back to itself 🔁CRF: Cannot rename file. Something went wrong while trying to rename a file.CSP: Cannot set property. Something went wrong while trying to change a file's properties. This could mean you tried to set a property only the file system is allowed to set, or a property that isn't defined on this type of file system.DAE: Directory already exists. A directory couldn't be created because a directory or file with this name already exists.DAT: Data error. The file system contains unreadable data. This could mean data errors detected by hardware, or inconsistent data inside the file system 💾DEV: Device not found. The device of the file wasn't found or doesn't exist.DND: "Do Not Delete" flag set. Someone tried to delete a file that's marked with a "Do Not Delete" flag 🚫DNE: Directory not empty. An invalid deletion of a nonempty directory was attempted.DNF: Directory not found. The directory wasn't found or doesn't exist. This refers specifically to the containing directory. If you're trying to access a directory and the actual directory you want isn't found, FNF (for File Not Found) should be shown instead.EPC: Error pending on channel. The server can't close the channel while it's trying to close or finish it.FAE: File already exists. The file couldn't be created because a file or directory with this name already exists.FNF: File not found. The file wasn't found in the containing directory. The TOPS-20 and TENEX "no such file type" and "no such file version" errors should also report this condition 🧚FOO: File open for output. Someone tried to open a file that was already opened for output.FOR: Filepos out of range. Someone tried to set the file pointer past the end-of-file position, or to a negative position.FTB: File too big. The file is larger than the maximum file size the file system supports.HNA: Host not available. The file server or file system is intentionally denying service to the user. This doesn't mean the network connection failed. It means the file system is explicitly not available 🙅IBS: Invalid byte size. The value of the "byte size" option wasn't valid.ICO: Inconsistent options. Some of the options given in this operation clash with others.IOD: Invalid operation for directory. The specified operation is invalid for directories, and the given pathname specifies a directory, in directory pathname as file format.IOL: Invalid operation for link. The specified operation is invalid for links, and this pathname is the name of a link 🔗IP?: Invalid password. The specified password was invalid 🔑IPS: Invalid pathname syntax. This includes all invalid pathname syntax errors.IPV: Invalid property value. The new value provided for the property is invalid.IWC: Invalid wildcard. The pathname isn't a valid wildcard pathname.LCK: File locked. The file is locked and can't be accessed, possibly because another process is using it 🔐LIP: Login problems. Something went wrong while trying to log in to the file system.MSC: Miscellaneous problems. A grab bag 🎒NAV: Not available. The file or device exists but isn't available. Typically the disk pack isn't mounted on a drive, the drive is broken, or something similar. Operator intervention is probably needed to fix it, but retrying is likely to work once the problem is solved 🛠️NER: Not enough resources. For example, a system limit on the number of open files or network connections has been reached.NET: Network problem. The file server had some sort of trouble creating a new data connection, or performing some other network operation, and couldn't do it 📡NFS: No file system. The file system wasn't available. For example, this host doesn't have any file systems, or its file system can't be initialized or accessed for some reason, or the file system simply doesn't exist.NLI: Not logged in. A file operation was attempted before logging in. Normally the file system interface always logs in before doing any operation, but this can happen in certain unusual cases where logging in was aborted.NMR: No more room. The file system is out of room. This can mean any of several things:- The entire file system is full.
- The particular volume involved is full.
- The particular directory involved is full.
- The user's allocated quota has been exceeded.
RAD: Rename across directories. The devices or directories of the initial and target pathnames aren't the same, but on this file system they're required to be.REF: Rename to existing file. The target name of a rename operation is the name of a file that already exists.UKC: Unknown operation. An unsupported file system operation was attempted, or an unsupported command was attempted.UKP: Unknown property. The property is unknown.UNK: Unknown user. The specified user name is unknown to this host 👀UUO: Unimplemented option. An option to a command isn't implemented.WKF: Wrong kind of file. This includes errors where an invalid operation for a file, directory, or link was attempted.WNA: Wildcard not allowed.
11. TOKEN LIST TRANSPORT LAYER 🧚
PURPOSE: The Token List Transport Layer is a protocol that makes it easy to send simple structured data, like lists 💜
11.1. Introduction to the Token List Transport Layer
the Token List Transport Layer is a general-purpose protocol, so it's giving versatile 🦋 It sends "tokens" through its underlying stream. Each token usually represents a simple quantity, like a string or an integer.
Tokens can be organized into "token lists." Special tokens mark where a list starts and where it ends. The layer tells apart "top-level token lists," which aren't contained in other lists, and "embedded token lists," which are contained inside other lists. Lists are super convenient for sending structured records, like the commands and command responses of the client protocol 🎀 And top-level token lists are what give things robustness ✨
The Token List Transport Layer is an umbrella term covering two related subjects: the "token list stream" and the "token list data stream." The token list stream is commonly used by apps that can easily organize their info into tokens and lists. The token list data stream is better for a big pile of data that can't easily be structured into tokens and lists, like file data, which is just a sequence of characters or bytes 🌸
Here's the main difference between the two 👑
Token List Data Stream Token List Stream
---------------------- -----------------
Built on: token list stream Byte Stream with Mark
Transmits: stream data tokens, token lists
Example
of use: NFILE data channels NFILE control
connectionNFILE uses the Token List Transport Layer and is a perfect example of why it's so useful 🔥 The NFILE commands and command responses go over the control connection in a token list stream. The file data goes across each data channel in a token list data stream.
11.2. Token List Stream
11.2.1. Types of Tokens and Token Lists
All numbers in the token list documentation are written in decimal notation. Bytes are 8 bits long 🍓
TYPES OF TOKENS
Tokens are of the following types:
1. Atomic tokens ✨
okay so atomic tokens come in a few subtypes, bestie 💅
- Data tokens. A data token is a sequence of bytes with an effectively infinite maximum length 🌈 In some contexts a data token represents a string. In other contexts, it's just other arbitrary data.
- Each data token is preceded in the token list stream by a representation of its length in bytes.
- Data tokens that are under 200 bytes long are preceded by one byte containing their length in bytes. So a data token of 34 bytes gets preceded by one byte of value
34. - Data tokens 200 bytes or over are preceded by the byte known as
PUNCTUATION-LONG, of value201. After the201comes a four-byte-long number (least significant byte first) containing the length of the data token that follows.
- Numeric tokens. A numeric token is a sequence of bytes that represents and encodes a nonnegative binary integer. The largest valid integer is
2^63 - 1🔥- Numeric tokens are either short integers (less than
256) or long integers (greater than or equal to256). Short integers are preceded by the byte known asPUNCTUATION-SHORT-INTEGER, of value206. - Long integers are begun by
PUNCTUATION-LONG-INTEGER, of value207. One byte follows, containing the length (in bytes) of the long integer. The integer itself comes next, least significant byte first.
- Numeric tokens are either short integers (less than
- Keyword tokens. A keyword token is a sequence of bytes that represents and encodes a named identifier of the implemented protocol. The client protocol uses keyword tokens to convey a name, and the only significance of a keyword token is in its name 🧚
- Each keyword is preceded by the byte known as
PUNCTUATION-KEYWORD, of value208. The data token followingPUNCTUATION-KEYWORDrepresents the name of the keyword as a string. The characters are in upper-case standard ASCII.
- Each keyword is preceded by the byte known as
- Boolean truth. A special token that represents the Boolean truth value. It's known as
BOOLEAN-TRUTH, of value209<4> 💖
2. Control tokens 🎀
The token list stream supports four control tokens to delimit token lists, plus one padding token. Here's the whole crew:
TOP-LEVEL-LIST-BEGIN 202 This control token
appears at the start of
each top-level token list.
TOP-LEVEL-LIST-END 203 This control token
appears at the end of
each top-level token list.
LIST-BEGIN 204 This control token
appears at the start of
each embedded token list.
LIST-END 205 This control token
appears at the end of
each embedded token list.
PUNCTUATION-PAD 200 This padding token should
be ignored by the token
list stream. It can be
sent to fill buffers.TOKEN LISTS 🧚
A token list is a sequence of atomic tokens or token lists. Token lists are begun and ended by control tokens that delimit them. There are three types of token lists:
- Top-level token lists. These begin with
TOP-LEVEL-LIST-BEGINand end withTOP-LEVEL-LIST-END. Top-level token lists are not contained in other lists. - Embedded token lists. These occur inside other token lists. They begin with
LIST-BEGINand end withLIST-END. - The empty token list. This is a special example of the embedded token list. In some contexts, the empty token list represents Boolean falsity. An embedded empty token list is composed of a
LIST-BEGINfollowed immediately by aLIST-END. A top-level empty token list is composed ofTOP-LEVEL-LIST-BEGINfollowed immediately byTOP-LEVEL-LIST-END.
11.2.2 Token List Stream Example 🍓
This section is an example of some data that can appear on a token list stream. The example is a top-level token list encoding an NFILE DELETE command, and it's giving main character energy ✨
The DELETE command is made up of these pieces: a TOP-LEVEL-LIST-BEGIN, the keyword DELETE, a data token containing the transaction identifier, a LIST-BEGIN, a LIST-END, a data token containing a pathname of a file to be deleted, and a TOP-LEVEL-LIST-END. This example uses t105 as the transaction identifier, and /usr/max/temp as the pathname.
All numbers in this section are expressed in decimal notation 🔢
The pieces of the command are listed here in order:
TOP-LEVEL-LIST-BEGIN- The keyword token whose name is DELETE
- The data token containing the characters:
t105 LIST-BEGINLIST-END- The data token containing the characters:
/usr/max/temp TOP-LEVEL-LIST-END
Now let's translate each piece of the command into the bytes that get transmitted through the token list stream 💅
- TOP-LEVEL-LIST-BEGIN
202 represents TOP-LEVEL-LIST-BEGIN- The keyword token whose name is DELETE.
A keyword token is introduced by PUNCTUATION-KEYWORD, which is represented in the token list stream as the byte 208.
A data token follows, containing the string "DELETE". A data token under 200 bytes long is introduced by one byte containing its length in bytes. The length of this data token is 6 bytes.
The data token continues with the standard ASCII character set representation of each character in the string DELETE:
208 represents PUNCTUATION-KEYWORD
006 represents the length of this data token
068 represents "D"
069 represents "E"
076 represents "L"
069 represents "E"
084 represents "T"
069 represents "E"- The data token containing the characters: t105
This data token is begun by its length in bytes (4), and continues with the NFILE character set representation of each character in the string:
004 represents the length of this data token
116 represents "t"
049 represents "1"
048 represents "0"
053 represents "5"- LIST-BEGIN
204 represents LIST-BEGIN- LIST-END
205 represents LIST-END- The data token containing the characters: /usr/max/temp
013 represents length of this data token
047 represents "/"
117 represents "u"
115 represents "s"
114 represents "r"
047 represents "/"
109 represents "m"
097 represents "a"
120 represents "x"
047 represents "/"
116 represents "t"
101 represents "e"
109 represents "m"
112 represents "p"- TOP-LEVEL-LIST-END
203 represents TOP-LEVEL-LIST-END11.2.3 Mapping of Lisp Objects to Token List Stream Representation 🌸
The Symbolics interface to the token list stream sends Lisp objects through the underlying Byte Stream with Mark, and produces Lisp objects on the other end. Not all Lisp objects can be sent this way. For example, compound objects other than lists are not handled. An appropriate analogy is the sending and reconstruction of list structure via printed representation 🙈
These are the types of objects that can be sent, and how they're represented:
- Lisp strings are represented as data tokens in the NFILE character set. Only 8-bit strings can be sent <5>.
- Keyword symbols are represented as keyword tokens. Although they're identifiable and reconstructable as keyword symbols, only their names are sent. Any properties, bindings, and the like are not sent.
- T is represented as
BOOLEAN-TRUTH. - NIL is represented as the empty token list.
- Lists are represented as token lists. Circular lists cannot be sent. See the footnote related to the ambiguity between NIL and the empty list: See the section "Types of Tokens and Token Lists", section 11.2.1 😭
- Integers are represented as numeric tokens. Only nonnegative integers less than
2^63can be sent 🦋
11.2.4. Aborting and the Token List Stream ✨
Okay so a token list stream gets all the perks of the abort management policy from the Byte Stream with Mark it's built on 💖. To actually get that benefit, any implementation of the token list stream has to follow some simple rules 🎀.
When the spec says "transmission," it means either one atomic token or one complete top-level token list 🦋. A transmission starts with the control token TOP-LEVEL-BEGIN and ends with TOP-LEVEL-END. The top-level token list can contain embedded token lists inside it 🧚.
The interface that writes to the token list stream has to be able to write out entire transmissions 💅. When it gets called, it has to effectively lock the token list stream and keep other processes out until the whole transmission has been encoded and sent.
If sending gets aborted while the stream is locked, the stream goes into an "unsafe" state 😭. Trying to send data while it's unsafe signals an error. To recover, the application and the token list stream must send a mark to cause resynchronization, and then the stream can be used again 🫶. When the reading side hits that mark, it resynchronizes itself according to whatever client protocol is in use.
Reading works the same way, so the interface that reads from the token list stream must be able to read entire transmissions too 📖. When it's called, it must lock the stream and keep other processes out until the whole transmission has been read.
If reading gets aborted while the stream is locked, the stream enters an unsafe state. The only way out of that unsafe state is by receiving a mark 🚪. While the stream is unsafe, the only valid operation is "read and discard all tokens until a mark is encountered; read and discard that mark; declare the stream safe again".
Depending on the client protocol, receiving a mark might make the reading side go looking for further marks. NFILE implements resynchronization of token list streams and is a super useful example, so check out the section "NFILE Control Connection Resynchronization", section 9.1 🌸.
The Symbolics implementation gives you two mark-handling primitives, and they work like this 🔥:
- Send a token (or list) preceded by a mark. When the stream is in the unsafe state (on the output side), this is the only permitted output operation (other than closing).
- Read through to a mark and read the token (or list) following the mark. When the stream is in the unsafe state (on the input side), this is the only permitted input operation (other than closing).
11.3. Token List Data Stream 🍓
The token list data stream is a facility for sending stream data through a token list stream. It imposes the following protocol on the data it transmits:
- Data is sent in the format of loose data tokens, not contained in token lists.
- The keyword token EOF indicates that the end of data has been reached.
- Token lists can be transmitted through the token list data stream.
- No loose tokens other than data tokens or the keyword token EOF can be sent.
- Boundaries between data tokens are not signification. The data is considered to be a continuous stream, with the possible exception of marks.
The token list data stream is most appropriate for sending file data 📁. It is expected (but not required) that its typical mode of use is to send a large number of data tokens, with an occasional token list. The design intent was that token lists would be used by the application program to indicate exceptional situations ⚠️.
Data tokens, the keyword token EOF, and token lists are all defined in the token list stream documentation, so see the section "Types of Tokens and Token Lists", section 11.2.1 💖.
NFILE is a great example of token list data streams in action. NFILE sends file data through token list data streams, and each NFILE data channel is a token list data stream 🦋. Errors such as disk errors during the reading of a file are conveyed as token lists through the token list data stream, so the errors ride along in the same pipe 🙈.
12. BYTE STREAM WITH MARK ✨
PURPOSE: Byte Stream with Mark is a simple layer of protocol that guarantees an out-of-band signal can be transmitted in the case of program interruption. It's designed to provide end-to-end stream consistency in the face of user program aborts 🎀.
12.1. Discussion of Byte Stream with Mark 🦢
INTRODUCTION
Byte Stream with Mark is a reliable, bidirectional byte stream with one out-of-band (but not out-of-sequence) signal called a "mark" 💜. The design ensures that the mark is always recognizable on the receiving end. Byte Stream with Mark is built on an underlying stream, which must support the transmission of 8-bit bytes. It has been implemented to run on TCP and Chaos, and marks are implemented differently on the two protocols 🌈.
Marks are used to resynchronize the stream when something has interrupted normal operations. For example, an application layer sending data over Byte Stream with Mark can abort in the middle of sending that data. Recovery is handled by sending a mark 🫶.
In the context of this document, "aborting" is defined like this:
Aborting the current execution of a program means to halt that execution and to abandon it, never to complete it. The data representing the state of the execution are irrevocably discarded. 💀
EXAMPLE OF USE
Byte Stream with Mark is the layer of protocol underlying NFILE 👑. NFILE uses the marks implemented in Byte Stream with Mark to resynchronize control connections or data channels whose synchronization has been lost. For a description of NFILE's use of marks to resynchronize streams, see the section "NFILE Resynchronization Procedure", section 9.
BYTE STREAM WITH MARK ON CHAOSNET
A mark is recognized on Chaosnet by a packet bearing the opcode 201 (octal). There is no data in a mark packet, so the data portion of the packet is ignored. Byte Stream with Mark transmits all data in packets bearing opcode 200 (octal) ⭐.
If Byte Stream with Mark is implemented on another (non-Chaos) stream that supports opcode-bearing packets, the recommended implementation is to reserve an opcode for the mark 🧚.
BYTE STREAM WITH MARK ON TCP: RECORD MODE
The whole point of Byte Stream with Mark is to guarantee that marks can always be unambiguously identified 🎯. So for TCP (and for any transport layer that doesn't implement packets natively), a simple record stream is imposed on the stream. The record boundaries only serve to distinguish where a mark can occur. A record consists of a two-byte byte count, most significant byte first, followed by that many bytes of data. A byte count of zero is recognized as a mark 🍓.
Both the sending side and the receiving side must rigorously maintain the integrity of the record boundaries 🔒. A writer to the stream must never output a byte count without that number of data bytes following. Similarly, a reader of the stream, after reading a byte count, has effectively contracted to read that many bytes from the encapsulated stream, regardless of whether those bytes are requested by the application layer 💅.
MAINTAINING RECORD INTEGRITY
This subsection deals with maintaining record integrity on non-Chaos networks. Since Chaos implements packets natively, no special care is required to maintain record integrity on the Chaos network 🦋.
The design discussed here guarantees record integrity, but the underlying stream must guarantee data integrity.
The basic design of Byte Stream with Mark on TCP (and other transport layers that don't implement packets natively) is to preserve record integrity by putting clearly demarcated, byte-counted records in the natural records of the encapsulated stream. Therefore, when the outer stream requests a buffer's worth of file data from the encapsulated stream, it expects to receive a buffer containing one entire, ntegral, record of that stream, complete with byte count 📦.
Because of diverse network implementations on different operating systems, the software that implements the encapsulated stream might not be able to provide integral record buffers to the Byte Stream with Mark implementation 😵. For example, the writing stream could have written records that are much longer than the available buffers on the receiving system. In this case, a request to read from the encapsulated stream returns some buffer or some amount of data representing less than an entire Byte Stream with Mark record. The input subroutine of the Byte Stream with Mark implementation must therefore return a region of this (smaller) buffer, representing less than the full Byte Stream with Mark record. Nevertheless, Byte Stream with Mark must extract the count of the full Byte Stream with Mark record from the first such buffer of each Byte Stream with Mark record, and maintain and update this count as succeeding component buffers are read 🧮.
In this case, if the program reading from Byte Stream with Mark aborts while reading data, the implementation of Byte Stream with Mark must continue to read through the remaining buffers of the Byte Stream with Mark record that has been subdivided in this fashion 🏃♀️.
The user side program will have determined that an abort has occurred, and will request Byte Stream with Mark to read up to and through the next mark. Byte Stream with Mark will have processed a fractional record, so it must discard the remaining buffers of the record now being read 🧹.
12.2 Byte Stream with Mark Abortable States 💅
Okay bestie, Byte Stream with Mark is built to keep the stream consistent end-to-end even when a user program gets aborted 🛑 ✨ That's the whole point! Quick definition first: "aborting" means you halt a program's execution and abandon it, so it never finishes. The data holding that execution's state is irrevocably discarded, gone forever 💀
USER PROGRAM ABORTS AND I/O STREAMS 🌸
Aborting code that manipulates I/O streams is genuinely a big problem 😭 A stream is a static data object that's meant to be used over and over, so aborting any routine working on one can leave it inconsistent and unusable. Not cute.
Lots of operating systems dodge this by doing most of their stream work inside the supervisor or executive program, which can't be aborted unless the whole system or network fails. Still, we need streams outside the supervisor too. And the Symbolics computer environment has no supervisor or executive program at all, so it's vulnerable to aborts everywhere 🙈
BYTE STREAM WITH MARK HANDLING OF USER PROGRAM ABORTS 🦋
Byte Stream with Mark is designed to be nearly impervious to aborts. The design comes from a careful analysis of every possible state of the stream and what an abort would do in each one. This section is that analysis, and it's honestly iconic 👑
A "transmission" is a chunk of user data sent by the application level through Byte Stream with Mark, and its end is well-defined once its start has been recognized. For example, when the token list stream uses Byte Stream with Mark, it sends token lists. Once a TOP-LEVEL-LIST-BEGIN has been sent, the containing transmission isn't considered complete until the matching TOP-LEVEL-LIST-END is read. See the section "Token List Transport Layer", section 11 💜
Here are the possible stream states when an abort happens:
- Abort occurs when the user program is not manipulating the stream.
This case presents no problem. ✨
- Abort occurs after a transmission has been partially sent, at a packet or record boundary.
This means the datum that would signal the transmission was completely sent hasn't been sent yet 🫶
The Byte Stream with Mark state is still consistent, but the application level state is not. The application level has to figure out that the code composing and sending its transmission was actually aborted, then start resynchronization using marks. 🔄
The receiving side must be careful not to act on a transmission (meaning, not perform any action or side effect) until the whole transmission has been successfully received. That's how the user program stays safe even if an abort lands after only part of a transmission was sent 🛡️
- Abort occurs during the sending or receiving of a record.
This is the most vulnerable state of the whole mechanism 😬 This case doesn't happen on packet-oriented media, since the next case covers it.
It's handled by minimizing how big this window is, and by killing the connection when the situation is detected. Depending on the operating system, the window could be shrunk with interrupt-disabling mechanisms, auxiliary processes or tasks, or some other technique ⚡
For buffered streams, input and output waiting can happen in consistent states, which keeps the time spent manipulating the actual encapsulated stream super short. For unbuffered streams, a lot of time can get spent in this window. Unbuffered streams are expected to be exceedingly uncommon, though. Even so, the implementation of Byte Stream with Mark must detect this case 🔍
- Abort occurs during the sending or receiving of fundamental units of the lowest-level underlying stream (packets, buffers, or bytes).
This case is usually handled by inhibiting interrupts, or using some other form of masking, in the code implementing the encapsulated stream. Nothing can wait at unexpected times, so that's the move 🙅♀️
13. POSSIBLE FUTURE EXTENSIONS 🌈
NFILE was designed to grow along with the needs of its clients, or with new clients who need different things 🧚 Right now it meets the needs of the Symbolics Genera 7.0 operating system, but the design is intentionally general. If users of other operating systems find new features that would be useful, they could be added to NFILE. This section shows some areas where NFILE's design deliberately makes room for extensions 💖
- The NFILE protocol encodes commands and responses as text, instead of using prearranged numbers. That means new commands and responses can be added without getting a new number from a central registry. 📝
- The Token List Transport Layer provides a general substrate for the value-transmission portion of network protocols. It's actually been used at Symbolics for other protocols besides NFILE. The Token List Transport Layer could conveniently be extended to transmit other types of values besides the ones it supports right now. 🍓
- The character set used for file transfer could be made negotiable. 🎀
- The command character set could be made negotiable too. There's no negotiation sequence yet, but one could be added. 💅
- Greater support for more complex file organizations could be added, like record files, databases, and so on. This could be an extension to the direct access mode facility. 🗂️
- Right now, the
LOGINcommand lets the user side tell the server which version of NFILE it's running. This is included so a server can keep supporting older versions of the protocol even after newer, extended versions exist. However, the specification is currently a bit vague about how the server can actually use the version info 🤔 - NFILE isn't restricted to TCP or Chaos as its underlying protocol. It can be built on any byte stream protocol that supports reliable transmission of 8-bit bytes and multiple connections. 🦋
We also want to mention one known limitation of NFILE, not just future extensions, so here it is 🥺
Right now NFILE requires multiple connections for a single session. The control connection has to be separate from the data connections. If NFILE is going to be used over a telephone, that requirement is a pretty inconvenient restriction 📞 It's possible to implement a multiplexing scheme as a layer between NFILE and the communication medium, though 🌸
APPENDIX A
NORMAL TRANSLATION MODE 🎀
NORMAL translation mode guarantees the following:
- A file containing characters in the NFILE character set can be written to any NFILE server and read back intact, containing the same characters. 💜
- A file written by NFILE should not appear as "foreign" to a server operating system unless the file contains NFILE's extended characters. In other words, a server file that uses only the subset of the NFILE character set limited to standard ASCII characters (the 95 printing characters, plus the native representation of return, linefeed, page, backspace, rubout, and tab) can be read and written, and the result is the same data in NFILE characters as exists in server characters. ✨
In this section, all numbers designating values of character codes are to be read as octal. The notation "x in c1..c2" means "for all character codes x such that c1 <= x <= c2." 🔢
The NFILE character set is an extension of standard ASCII. The 95 ASCII printing characters have the same numerical codes in the NFILE character set. Five ASCII non-printing characters have counterparts in the NFILE character set, shown in the table below 👀 The NFILE character set includes a single Return character, instead of the carriage-return line-feed sequence typically used in ASCII. The NFILE character set doesn't include the ASCII control characters, except for those five in the table, but it does include some extra printing and formatting characters that have no ASCII counterparts 🌈
NFILE Standard ASCII
Rubout: 207 177
Backspace: 210 10
Tab: 211 11
Linefeed: 212 12
Page: 214 14Heads up: the NFILE Return character is code 215 🚨 It includes "going to the next line." This is a notable difference from the convention used in PDP-10 ASCII, where lines end with a pair of characters, "carriage return" and "line feed" 💖
NORMAL TRANSLATION TO UNIX SERVERS 🐧
The translation in this table is appropriate for UNIX servers, or other servers that use 8-bit bytes to store ASCII characters. Machines with 8-bit bytes usually put the extra NFILE characters in the top half of their character set 🫶
TABLE 1. TRANSLATIONS FROM NFILE CHARACTERS TO UNIX CHARACTERS
NFILE character UNIX character
x in 000..007 x
x in 010..015 x + 200
x in 016..176 x
177 377
x in 200..207 x
x in 210..211 x - 200
212 015
x in 213..214 x - 200
215 012
x in 216..376 x
377 177
TABLE 2. TRANSLATIONS FROM UNIX CHARACTERS TO NFILE CHARACTERS
UNIX character NFILE character
x in 000..007 x
x in 010..011 x + 200
012 215
x in 013..014 x + 200
015 212
x in 016..176 x
177 377
x in 200..207 x
x in 210..215 x - 200
x in 216..376 x
377 177NORMAL TRANSLATION TO PDP-10 FAMILY SERVERS 🧠
The translation in this table is appropriate for PDP-10 family servers, or other servers that use 7-bit bytes to store ASCII characters. On the PDP-10, the sequence CRLF (015 012) represents a new line 🌷
The mechanism for this translation on machines with 7-bit bytes is to use the RUBOUT character (octal code 177) as an escape character 🔐
TABLE 3. TRANSLATIONS FROM NFILE TO PDP-10 CHARACTERS
NFILE character PDP-10 character(s) x in 000..007 x
x in 010..012 177 x
013 013
x in 014..015 177 x
x in 016..176 x
177 177 177
x in 200..207 177 x - 200
x in 210..212 x - 200
213 177 013
214 014
215 015 012
x in 216..376 177 x - 200
377 no corresponding codeokay so these tables look like a total maze at first, bestie 🫠 but there are a few general rules that make it click 💖 First, NFILE characters in the range 000..177 are generally just themselves ✨ and any x in 200..377 is generally written as 177 followed by x - 200. So 177 is basically the quote character that unlocks the second 200 NFILE characters 🎀 That's why 177 177 means 177, and there's no way to write 377 with PDP-10 ASCII characters. It was deemed that 177 is a way more useful and common character than 377, so 377 got the short end of the stick 🙈
In the NFILE character set, the formatting control characters are offset up by 200 compared to standard ASCII 🌈 This is why the preferred way to write 210 (backspace) is 010, and 010 turns into 177 010. Same energy for 211 (Tab), 212 (Linefeed), 214 (Formfeed), and 215 (Return) 💅
Now the Return character needs extra care 🫶 because it's the mapping of the system-dependent idea of "the start of a new line." The NFILE Return (215) is equivalent to 015 012 (CRLF) on some ASCII systems. In the NFILE character set there is no representation
TABLE 4. TRANSLATIONS FROM PDP-10 CHARACTERS TO NFILE CHARACTERS
PDP-10 character NFILE character
x in 000..007 x
x in 010..012 x + 200
013 013
014 214
015 012 215
015 not-012 115
x in 016..176 x
177 x in 000..007 x + 200
177 x in 010..012 x
177 013 213
177 x in 014..015 x
177 x in 016..176 x + 200
177 177 177of a carriage that doesn't go to a new line, so if there is one in a server file, it must be translated to something else. When converting ASCII characters to NFILE characters, an 015 followed by an 012 therefore turns into a 215. A stray CR is arbitrarily translated into a single M (115).
APPENDIX B ✨ RAW TRANSLATION MODE
RAW mode means no translation should be performed 🍓 In RAW mode, the server operating system should treat the file as a character file and use the same data formatting that would be appropriate for a character file, but transfer the actual binary values of the character codes 💜
APPENDIX C ✨ SUPER-IMAGE TRANSLATION MODE
SUPER-IMAGE mode is intended for use by PDP-10 family machines only 🧚 It's mostly here as an illustration of a system-dependent extension. A server machine that has 8-bit bytes should treat SUPER-IMAGE mode the same as NORMAL mode 🌸
In this section, all numbers designating values of character codes are to be interpreted in octal 🔢 and the notation "x in c1..c2" just means "for all character codes x such that c1 <= x <= c2."
SUPER-IMAGE mode suppresses the use of the 177 character as an escape character 🙅♀️ Character translation should be done as in NORMAL mode, with one exception. When a two-character sequence beginning with 177 is detected, the 177 should not be output at all.
In this section, all numbers designating values of character codes are to be interpreted in octal. SUPER-IMAGE mode is intended for use by PDP-10 machines only, so yes, everything numeric here is octal again 🔢
SUPER-IMAGE suppresses the use of Rubout for quoting 🎀 So for each entry beginning with a 177 in the PDP-10 character column of the NORMAL translation table, the NFILE character ends up with the 177 removed.
TABLE 5. SUPER-IMAGE TRANSLATION FROM NFILE TO ASCII
NFILE character PDP-10 character(s)
x in 000..177 x
x in 200..214 <x - 200>
215 015 012
x in 216..376 <x - 200>
377 no corresponding code TABLE 6. SUPER-IMAGE TRANSLATION FROM ASCII TO NFILE
PDP-10 character NFILE character
x in 000..007 x
x in 010..012 x + 200
013 013
014 214
015 012 215
015 not-012 115
x in <016..176> x
177 177NOTES 📝
- NFILE's requirement for using the NFILE character set is acknowledged as a real drawback for non-Symbolics machines 😤 A genuinely useful upgrade would be a way to make the character set negotiable 🦋
- Implementation note: Care must be taken that the freeing is done before the control connection is allowed to process another command, or else the control connection may find the data channel to be falsely indicated as being in use 🧠
- The Symbolics operating system has the policy that whenever the user side is waiting for the server side, a user abort can occur. This user side waiting can occur in any context, such awaiting a response, waiting in the middle of reading network input, or waiting in the middle of transmitting network output. Thus there are no "hung" states ✨
- Heads up, the Token List Transport Layer gives you a special token for Boolean truth, but there's no matching token for Boolean falsity 🙃 NFILE uses an empty token list to indicate Boolean falsity instead. The historical reason is the inability of the Lisp language to differentiate between the empty list and NIL, which is traditionally used to mean Boolean falsity 💀 If the flexibility of both a Boolean falsity and an empty token list were allowed, it would create problems for an operating system that cannot distinguish between the two. This aspect of the protocol is recognized as a concession to the Lisp language. The unfortunate effect is that operating systems are not allowed to tell Boolean falsity apart from an empty list 😭
- And finally, no so-called "fat strings" can be sent 🚫🍓
📦 where this page comes from
- ietf: rfc1037 1987-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms