RFC 304
Data Management System Proposal for the ARPA Network
RFC 304: A Data Management System Proposal for the ARPA Network 🎀
okay bestie, this is RFC 304 from the Network Working Group, written by D. B. McKay at IBM and dated February 17, 1972 (NIC 9077) 💅 It's filed under categories D3, D4, and D7. It obsoletes nothing and updates nothing, so it's totally standing on its own ✨
Introduction ✨
okay so this proposal exists to get the conversation started about designing a network Data Management System 🌸 It is not meant to be the complete, everything-included design for the ultimate protocol that lets users share data easily. It's more of a framework that helps us spot and build the tools we need in a unified way, so the network can manage its resources to the best advantage for the user 🦋
the whole point is not to try and solve an impossible problem. it's to bring a super necessary service to users so they can run applications they literally couldn't run before 😭 it's meant to be consistent with every other major function already built into the network, i.e., NCP - 2nd level protocol, Telnet, and the Form Machine 🫶 the Data Management Service, or Data Control Facility (DCF), does the same kind of thing, just at a higher level of application building on top of the tools that already exist in the network 🔥
data that gets referred to and transmitted in this system is treated as a special class called network data 💜 it's named and characterized through a network datalanguage, and all the pertinent info about where it can be located and what its structure is gets kept in a network catalog 📚 access to the data for its actual transmission goes through NCP socket addressable routines, kind of like how the SMFS at Santa Barbara works ✨ it's feasible that the SMFS will become an active resource that the DCF uses 👑
System Overview 🌈
there are six functionally and logically distinct areas you can identify in the Network Data Service (Figure 1), each with subfunctions that can be categorized and discussed 💖
- The user interface to the DCF. in an interactive environment like the ARPA network, this interface would be handled by Telnet, supporting the local user at their terminal and directing the request to the DCF. the DCF in this case would be a specialized server task 🎀
- The DCF, or the functional unit responsible for coordinating all the activity of the Network Data Service. it also houses the interfaces to all the other functions 🧚
- The Network Catalog, or Directory, which contains all the information about network data 📖
- The Data Reconfiguration Service, or Form Machine, which gets called on when data translation or reconfiguration is needed. this gets invoked automatically by the DCF when possible, which takes that responsibility off the user's plate 🙈 for more specialized translation, though, the user can still write programs for the Form Machine and execute them there 💅
- The remote DCF, or DCF', which contains enough function to recognize the request being made of it by the DCF. it's a server task to the DCF 🦋
- File xfer protocol, which is a function that the DCF and the DCF' initiate as the means to control data transfer in the network 🔥
more detailed discussion of each of these areas shows up in the following sections 🌸
User Interface 💜
it was stated in RFC146 that the DCF should handle all network resources as a single resource and use it as well as it can 👑 this statement was also meant to include the Data Computer and Unicon storage as part of that resource 🫶 how far this can actually go is an open question 🤔 but using the Data Language developed by CCA would give users a consistent interface to these network services, and it might also make it easier for the DCF to use the Data Compiler ✨
it should be pointed out right now that the DCF is a logical function that can reside anywhere, including on the Data Computer 🧚
the user should be allowed to enter all commands and updates interactively to the DCF 💖 the DCF will be a serving user process that interfaces with the Telnet Server routine 🎀 the actual data in the terminal transmissions will be the commands and data the user sends to the DCF 📡 by adopting Telnet as the initial user interface, every user with Telnet can access the DCF 🌈
the actual user commands and data are an area that needs more investigation 👀 the following comments offer suggestions for what a final data language and manipulation language should do 💅
there are at least two logically distinct functions that must be performed. one is the actual defining and redefining of the data. the other is the request for service, such as a catalog entry or a request for information. this proposal is not intended to provide access to every data base in the network 🙅♀️ instead it's aimed at the files that are catalogued and known to the DCF, in the same way the Data Computer knows its own Data Base 🌸
the following Data Description Concepts, described in detail in CCA's Data-language, are also useful in the DCF 🦋 first of all, Data Containers are groups of nested boxes. the box represents the data, or other data containers, kept inside it. it represents a named set of locations in some storage medium 📦 there are also several types of data container, such as STRING, INT, REAL, PTR, ARRAY, LIST, STRUCT, and MIX 💜 finally, each of these containers can be named. the name can also qualify the item of interest by concatenating names to reflect the logical nesting of the containers 🫶
although storage and retrieval mechanisms should be the same as the ones proposed in the DL, initially it should not be necessary to implement all the functions that filter and manipulate the data 🥰 for example, in an initial implementation of a DCF, it would not be necessary to give the user relational, boolean, and computational operators 🙈 users who are specifically interested in that kind of service could be directed to use the Data Computer on their own initiative, or as a service of the DCF ✨
several of the operators specified in the language are important and must be considered in the DCF 🔥
the following list represents key operators for a DCF, with a brief description of what each function is. for a more detailed discussion of each statement, the reader should read the Datalanguage report by CCA 📚
the assign operation places a value into the containers, where the containers are either single items or reference other containers 💖
subscripting allows selection by element number 🎯 it's a powerful tool for specifying, in large containers of data, the reference and transmission of only the parts that are necessary 🌈 files can be subsetted by containerization, referencing fields and records, which can be further classified by subscripting 🧚 this subscripting function can be extended even further to include the DL's virtual list concept 🦋
maintenance of the files must be provided with the delete and add functions applied to the container-referenced data 🛠️✨
Data Conversion (Form Machine) 💅
okay so the Form machine is an essential network function, and the DCF can invoke it whenever it's needed 💖. It gets used the same way it was always meant to be used, except now the DCF is the one doing the using instead of the user 🧚. That gives the user one common interface to the network. The user works with the Datalanguage for file transfer and conversions, and the DCF handles their Form machine services behind the scenes 👑.
The reason the Form machine makes sense is that a network data sharing facility should provide it. Application programs need different formats from program to program, so the network should adapt to each program's requirements 🎀. Console configurations and machine dependent data have the same problem 🌸.
The way it works is that application programmers write descriptions of their data as forms, and the service stores those forms by name ✨. In this case the DCF would run the data transformation on the network data stream by calling the forms by name. There would be a standard set of forms for machine dependent data, written as part of a general implementation of the DCF, and the DCF would invoke them when necessary 🦋.
There are three conceptual connections to the Data Reconfiguration Service (DRS) 🫶.
- The critical connection between the originating user and the DRS. In this case it would be the DCF. This raises the question of how the user would communicate with the Form Machine. They could use normal procedures and the directory to reach the DRS by Telnet, or they could let the DCF set up a connection for them automatically, with their defined forms cataloged in the network catalog 💅. 2.-3. The other two connections are between the user process and the serving process, and the DRS makes them through the NCP 🧚.
The Form Machine is such an invaluable service to the network that it is imperative it reside in several locations, with user named forms available at each DRS location. That way service is available when it's needed 💖. However, having the DRS invoked by the DCF raises two interesting areas of investigation 👀.
The first is the common interface the network presents to the user. If the Datalanguage is going to be the common interface, is it practical and feasible for the DCF to perform a mapping service that converts Datalanguage statements into the proper parameters and statements the Form Machine needs? This is an area that has to be discussed and investigated further. I'd really encourage anyone to submit an RFC on this subject 🍓.
The second question is a simple one, and it's about whether the file transfer protocol is sufficient to handle the Form Machine connections from a remote location 🤔.
File Transfer Protocol (FTP) 🦋
The goals of FTP are to promote sharing of files, encourage implicit (without explicit login) use of computers, and shield the user from variations in file and storage systems across hosts 💜. The DCF uses these related operations in FTP, which lets us stretch those ideas down to the restructuring and subfile level we talked about in the user interface section ✨.
Also, a request can be made to any DCF on any system to retrieve a file and transfer it to any other system 🌈. That means implicit use of systems is encouraged, and the user doesn't have to explicitly link their system to the system doing the transfer. Say someone is running an analysis program at BBN, which may also have a DCF. A request for a file to be shipped to BBN from the Santa Barbara 75 could be made to the DCF at BBN, and the user never has to talk to the 75 directly 🙈.
The sending and receiving connections are initiated by the DCF, and the logical link between the two systems obeys FTP 🧚. The only modification I can see being necessary in FTP is an acknowledgement to the commands the DCF sends to the sending and receiving sites. On top of that, an acknowledgement to the end of file indication would go to the sending system and to the DCF 💅. The rename from, rename to, delete and list requests are transmitted by the DCF directly, and all acknowledgments come back to it 🫶.
The remote DCF and DCF' from earlier would recognize and handle all the FTP messages 🍓. They would also recognize requests for a particular container or subset of the data. They should be able to recognize the information given to them, access the requested data, and strip off the necessary information and transmit it 🔥.
How complex the DCF' gets depends on how much functional capability is built into the network portion of the Data-language 👑.
Conclusion 🎀
This paper is meant to spark ideas and discussion in all of the areas we covered 💖. The main goal is to kick off a coordinated specification and implementation effort to provide data sharing in the network 🌸.
,---------------.
| |
| DCF' |
Sending | |
Host `---------------'
\ ^ /
\_____________|__________/
|
/ | ^
File / | |
Xfer / |
Protocol / DRS | |
/ Protocol |
/ | |
/ |
,----------. /\ ,---------./ /\ ,------------|--.
| | / \ | | / \ | |
| TELNET |---\ / ---| DCF |---\ / ---| DRS | |
| | \/ | | \/ | |
`----------' | |\ `------------|--'
^ `---------' \File |
| ^ | \Xfer DRS | |File
| | | \Protocol Protocol | Xfer
v | v \ | |Proto-
,----------. ,---------. \ | col
| USER | | | \ | |
| TERMINAL | | CATALOG | \ | v
| | | | \ _______________________
`----------' `---------' / | \
/ v \
Receiving ,---------------.
Host | |
| DCF' |
| |
`---------------'
Network Data Service
System Overview
Figure 1The keyword statements of the language matter a lot for data manipulation and transfer 🧚. These keywords start entering information into the net catalog and access the physical data sitting at the various systems 💅. Most of these keyword commands would be sent to the remote systems as part of the file transfer protocol 🦋.
Some examples of the keywords the DL uses are CREATE, DELETE, OPEN, TRANS, CLOSE, and DEFINE ✨.
Network Catalog and Directory 🌸
The actual structure of the network catalog should be pretty straightforward 💖. It will hold all the information needed to retrieve data files and their designated subsets 📚. At the start, the catalog doesn't need to hold everything we'd love it to, like authorization for use, access, or update. So it's really important that the catalog is an open-ended structure that's easy to add to 🫶.
The main job of the network catalog is to store all the network data file structure information the user has entered through the Data-language 💅. It must also say how the user's logical description of a file is tied to the actual physical file and its location. That physical information must include the right pointers and addresses to actually retrieve the data 🔥. Since the files we're dealing with are network files, accessible to network users through functions like SMFS, the addressing information can be pathnames, as suggested in the Rename Convention in the file transfer protocol 🌈.
[ This RFC was put into machine readable form for entry ] [ into the online RFC archives by Mirsad Todorovac 11/98]
📦 where this page comes from
- ietf: rfc304 1972-02
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms