RFC 610
Further datalanguage design concepts
RFC 610: Further Datalanguage Design Concepts 💖
Title block, bestie: this is RFC # 610, from the Network Working Group 🌸, written by Richard Winter, Jeffrey Hill, and Warren Greiff of the Computer Corporation of America (CCA) ✨. Its NIC # is 21352, and it's dated December 15, 1973 📅. The title block doesn't list a category or status, and it doesn't say this one obsoletes or updates anything, so there's nothing like that to track 🫶.
Acknowledgment 🎀
During the Datacomputer Project, a lot of people helped build up datalanguage 🧚.
Dr. Gordon Everest (University of Minnesota), Dr. Robert Taylor (University of Massachusetts), Professor Thomas Cheatham (Harvard University) and Professor George Mealy (Harvard University) gave suggestions and criticisms that were especially useful 💜.
Inside CCA, the authors weren't the only ones in the language design at different stages of the project 💅. Hal Murray, Bill Bush, David Shipman and Dale Stern were super helpful too 🙌.
1. Introduction ✨
1.1 The Datacomputer System 🖥️
Okay so the datacomputer is a large-scale data utility system, meaning it offers data storage and data management services to other computers 💾.
It's different from traditional data management systems in a few ways, and honestly, they're kind of iconic 👑.
First, it runs on dedicated hardware. It's its own computing system, specialized just for data management 🎀.
Second, it's built on a large scale. Data is meant to live on mass storage devices with capacities around a trillion bits 🌈. Files on the order of one hundred billion bits are supposed to be kept online ⭐.
Third, it's meant to support sharing of data among processes running in very different environments 🦋. The programs sharing a given database might be written in different languages, run on different hardware under different operating systems, and serve end users with radically different needs 😭. So to make that shared use work, transformations between hardware representations and data structuring concepts have to be achieved 🔥.
Finally, the datacomputer is designed to work smoothly as one piece of a much bigger system: a computer network 🌸. In a computer network, the datacomputer is a node that specializes in data management and acts as a data utility for the other nodes 💅. The Arpanet, which the datacomputer is being developed for, is an international network with over 60 nodes 🌍. Some of those nodes are specialized for terminal handling, some for computation (for example the ILLIAC IV), some are general purpose service nodes (for example MULTICS), and one (CCA) is specialized for data management 💖. That one is giving main character energy 👑.
1.2 Datalanguage 📜
Datalanguage is the language in which all requests to the datacomputer are stated 💌. It includes facilities for data description and creation, for retrieval of or changes to stored data, and for access to a variety of auxiliary facilities and services 🧚. With datalanguage, you can specify any operation the datacomputer is capable of performing 🤩. It's the only language the datacomputer accepts, and it's the exclusive means of access to data and services 🔐. No cap, it's the only door in 🚪.
1.3 Present Design Effort 🛠️
Right now we're working on complete specifications for datalanguage. This is the second iteration in the language design process 🔁.
A smaller, earlier design effort came up with some concepts and principles, which are described in the third working paper in this series 📚. Those became the basis of software implementations, which gave us an initial network service capability 🌐. A user manual for that system was published as working paper number 7 📖.
Experience from implementation and service, further study of user requirements, work with potential users, and investigation of other work in the data management field have all led to quite a few ideas for improving datalanguage 💡. Those ideas are being absorbed into the language design in the iteration that's in progress right now 🦋.
When the language design is complete, it'll be built into the existing software 🎁. That means changes to the language compiler, but very little impact on the rest of the system 🫶.
Datacomputer users will get their first access to the new language during 1975 🗓️. Mark your calendars, bestie 💜.
1.4 Purpose of this Paper 🎯
This paper shares concepts and preliminary results, not a finished design 🧁. There are two reasons for publishing it now 🙈.
The first is to give information to people planning to use the datacomputer 📬. They might benefit from knowing what we intend for its development 🌷.
The second is to let system and language designers comment on our work before the design is frozen 🧊. We want your feedback while it's still soft 🥹.
1.5 Organization of the Paper 🗂️
The rest of the paper is split into four sections 🍓.
Section 2 covers the most global considerations for language design 🌍. This is our view of the problem, it has shaped our work so far, and it'll determine most of our actions in completing the design 💅. It sets up the background for Section 3 and reviews some material that will feel familiar to anyone who's been following our work closely 👀.
Section 3 digs into some of the specific issues we've worked on 🔍. The emphasis is on solutions and options for solutions 💡.
In Sections 2 and 3, we're presenting our "top-down" work 🔝. This is the thinking we've done based on known requirements and our idea of the desirable properties of datalanguage 🧠.
We've also been working from the opposite end, building the primitives the language will be constructed from 🧱. Section 4 presents that work: a model datacomputer that will ultimately provide a precise semantic definition of datalanguage 🤓. Section 4 explains the part of the model that's complete, and relates it to our other work 🔗.
Section 5 discusses the work that remains, both on the model and in our top-down analysis 🚧.
2. Considerations for Language Design 🧩
2.1 Introduction 🌟
Data management is the task of managing data as a resource, independent of hardware and applications programs 💅. It can be divided into five major sub-tasks 🍰:
(1) _creating_ databases in storage,
(2) making the data _available_ (e.g., satisfying queries),
(3) _maintaining_ the data as information is added, deleted and
modified,
(4) assuring the _integrity_ of the data (e.g., through backup and
recovery systems, through internal consistency checks),
(5) _regulating_access_, to protect the databases, the system, and
the privacy of users.These are the major data-related functions of the datacomputer 💖. The system will eventually offer other services too (like accounting for use and monitoring performance), but those are really auxiliary and common to all service facilities 🧃.
This section lays out global considerations for designing datalanguage 🌸. They're based on our observations about the problem and the environment where it's going to be solved 👀. The central problem is data management, and the datacomputer shares the same goals as lots of currently available data management systems 🫶. Still, several aspects of the datacomputer create a unique set of problems to solve 🔥.
2.2 Hardware Considerations 🔧
2.2.1 Separate Box 📦
The datacomputer is a complete data management utility in a separate, closed box 🔒. The hardware, the data and the data management software are all segregated from any general-purpose processing facilities 🛡️. There's a separate installation dedicated to data management 🏠. Datalanguage is the only means users have for communicating with the datacomputer, and the datacomputer's sole activity is processing datalanguage requests 💌.
Dedicating hardware has an obvious advantage: you can specialize it for data management 🎯. The processor(s) can be modified to have data management "instructions", and common low-level software functions can be built into the hardware 🤖.
There's a less obvious, but possibly more significant, advantage that comes from the separateness itself 🤯. The system can be protected more easily 🔐. A fully developed datacomputer that only has maintenance activity can provide a very carefully controlled environment 🧼. First, it can be made as physically secure as required 🏰. Second, it only needs to run system software developed at CCA 💻. All user programs are written in a high-level language (datalanguage) that the system effectively interprets 🎀. So only datacomputer system software processes the data, and the system isn't very vulnerable to capture by a hostile program 🛑. Since there's potential to develop data privacy and integrity services that aren't available on general-purpose systems, we can expect less difficulty developing privacy controls (including physical ones) for the datacomputer than for the systems it serves 🥰.
2.2.2 Mass Storage Hardware 💿
The datacomputer will store most of its data on mass storage devices, which have distinctive access characteristics ⚙️. Two examples are Precision Instruments' Unicon 690 and Ampex Corporation's TBM system 🧲. They're quite different from disks, and they differ significantly from one another 🙃.
Almost all users will be clueless about the characteristics of these devices, though 🙈. Plenty of them won't even know the data they're using lives at the datacomputer 🫠. Also, as the system develops, data may be invisibly shunted from one datacomputer to another 🌀. So it can end up stored in a physical format that's quite different from the one it started with 😵💫.
In an environment like that, it's clear that requests for data should be stated in logical, not physical, terms 💡. Logical is the way to go, bestie 💜.
2.3 Network Environment 🌐
The network environment adds more requirements for datacomputer design 📡.
2.3.1 Remote Use 🛰️
Since the datacomputer is accessed remotely, the need for effective data selection techniques and good mechanisms for expressing selection criteria gets amplified 🔊. That's because of the narrow path network users have to communicate with the datacomputer 🚿. Right now, a typical process-to-process transfer rate over the Arpanet is 30 kilobits per second 🐢. That can go up through optimization of software and protocols, and through extra spending on hardware and communication lines 💸. Still, it seems safe to assume it won't soon approach local transfer rates (measured in megabits per second) 🏎️.
A typical request calls for either transferring part of a file to a remote site, or making a selective update to a file already stored at the datacomputer 📂. In both situations, good mechanisms for specifying which parts of the data to transmit or change will cut down the amount of data normally transferred ✂️. That matters a lot, because storing data at the datacomputer is cheap per bit, so transmission costs will be a big part of the total cost of using the datacomputer 💰. Not me being shook by the math 😭.
2.3.2 Interprocess Use of the Datacomputer System 🤝
Using the network effectively means groups of processes, far away from each other, have to be able to cooperate to get a task done or provide a service 🫂. For example, say a problem involves array manipulation, data retrieval, interaction with a user at a terminal, and the generalized services of a language like PL/I 🧮. It might be most economical to have four cooperating processes 🍕. One could run at the ILLIAC IV, one at the datacomputer, one at MULTICS, and one at a TIP 🌈. Setting up those four processes and having them communicate has overhead, sure, but each one is doing its job on a system specialized for that job 🎀. In lots of cases, using the specialized system gains you several orders of magnitude in economy or efficiency (for example, online storage at the datacomputer has a
capital cost two orders of magnitude lower than online costs on conventional systems) 💸 so ngl there is a lot of incentive to look at solutions where cooperating processes run on specialized systems. It's giving teamwork, and I'm obsessed 🫶
To summarize: the datacomputer must be ready to work as a component of small networks of specialized processes, so that it can be used effectively in a network where there are many specialized nodes. 🧚
2.3.3 Common Network Data Handling ✨
A large network can support enough data management hardware to build more than one datacomputer. Sure, that hardware could be combined into one even bigger datacomputer, but there are real advantages to setting it up as two (or possibly more) systems 🌸 Each system should be big enough to get economies of scale in data storage and to support the data management software. Important databases can be duplicated, with a copy at each datacomputer, so if one datacomputer fails, or gets cut off by a network failure, the data is still available 🛟 Even if duplicating the file isn't worth it, the description can be kept at the different datacomputers. That way, applications that need to store data constantly can be guaranteed that at least one datacomputer is available to receive input 💖
These kinds of failure protection involve cooperation between a pair of datacomputers. In a way, they require the two datacomputers to function as a single system 👯♀️ Given a system of datacomputers (which you can think of as a small network of datacomputers), it's obviously possible to experiment with extra services at the datacomputer-network level. For example, all requests could be addressed simply to the datacomputer-network. The datacomputer-network could then figure out where each referenced file is stored (i.e., which datacomputer), and how best to satisfy the request 🧠
So here, we've mentioned two kinds of cooperation in the network environment: cooperation among processes to solve a given problem, and cooperation among datacomputers to provide global optimizations in the network-level data handling problem. These are only two examples, and they're especially interesting because they can be implemented in the near term 🔥 In the network, much more general kinds of cooperation are possible, just a little farther in the future. For example, eventually, someone might want the datacomputer(s) to be part of a network-wide data management system, where data, directories, services, and hardware are all spread out around the network. The whole system could work as one unit under the right circumstances. Most requests would only use the data and services of a few nodes. Inside this network-wide system, there would be more than one data management system, but all of them would be interfaced through a common language. Because datacomputers are the biggest data management resource in the network, they would definitely play an important role in any network-wide system 👑 The language of the datacomputer (datalanguage) is certainly a convenient choice for the common language of such a system 🎀
So a final, albeit futuristic, requirement the network places on the design of the datacomputer system is that it be a suitable major component for network-wide data management systems. If it's feasible, we'd really like datalanguage to be a suitable candidate for the common language of a network-wide group of cooperating data management systems 🦋
2.4. Different Modes of Datacomputer Usage 🌈
Inside this network environment, the datacomputer is going to play several roles. In this section, four of those roles are described. Each one puts constraints on the design of datalanguage. We can analyze them in terms of four overlapping advantages the datacomputer provides:
- Generalized data management services
- Large file handling
- Shared access
- Economic volume storage
Of course, the main reason to use the datacomputer will be the data management services it provides 💅 However, for some applications, size will be the dominating factor, because the datacomputer will provide online access to files that are so large that only offline storage and processing used to be possible 😭 Sharing data between different network sites with wildly different hardware is another feature that only the datacomputer provides. And economies of scale make the datacomputer a viable substitute for tapes in applications like operating system backup 💾
Naturally, a combination of the above factors will be at work in most datacomputer applications. The following subsections describe some possible modes of interaction with the datacomputer ✨
2.4.1 Support of Large Shared Databases 🌍
This is the most significant application of the datacomputer, in nearly every sense. Like, it's THE main character 👑
Projects are already underway that will put databases of over one hundred billion bits online on the Arpanet datacomputer. Among these is a database that will ultimately include 10 years of weather observations from 5000 weather stations located all over the world 🌦️ As online databases, these are unprecedented in size. They'll be of international interest and will be shared by users working on a wide variety of hardware and in a wide variety of languages 🌸
Because these databases are online in an international network, and because they're expected to be of considerable interest to researchers in related fields, it seems obvious there will be extremely broad patterns of use 📈 So a strong requirement is a flexible and general approach to handling them. Giving different users of a database different views of the data is an overriding concern of the datalanguage design effort. It's discussed separately in Section 2.5 🫶
2.4.2 Extensions of Local Data management Systems 🧚
Picture local data handling systems (data management systems, applications-oriented packages, text-handling systems, etc.) that want to take advantage of the datacomputer 🦋 They might do this because of the economics of storage, because of the data management services, or because they want to use data that's already stored at the datacomputer. Either way, these systems have some distinctive properties as datacomputer users: (1) most would use local data as well as datacomputer data, and (2) many would be concerned with translating local requests into datalanguage 🤓
For example, a system that does simple data retrieval and statistical analysis for non-programming social scientists might want to use a census database stored at the datacomputer 📊 Such a system may perform a range of data retrieval functions, and it may need sophisticated interaction with the datacomputer. Its usage patterns would make a pretty big contrast with those of a single application program whose only use of the datacomputer is printing one specific report based on a single known file 💀
This social-science system would also use some local databases, which it keeps at its own site because they're small and more efficiently accessed locally. It would be really nice to think of data the same way whether it's stored locally or at the datacomputer 🥹 Certainly, at the lower levels of the local software, there will have to be differences in interfacing. Still, it would be lovely if local concepts and operations could easily be translated into datalanguage 💜
2.4.3 File Level Use of the Datacomputer 📁
In this mode of use, other computer systems take advantage of the online storage capacity of the datacomputer. To these systems, datacomputer storage is a brand new class of storage: cheaper and safer than tape, and nearly as accessible as local disk 🍓 Maybe they even automatically move files between local online storage and the datacomputer, so users get the impression that everything is stored locally online 🪄
The distinctive feature of this mode of use is that the operations are on whole files 🎀
A system operating in this mode only uses the ability to store, retrieve, append, rename, do directory listings and the like. An obvious way to make this file level handling easily available to the network community is to make use of the File Transfer Protocol (see Network Information Center document #17759 -- File Transfer Protocol), which is already used for host to host file transfer 📨
Although this "whole file" usage of the datacomputer would be motivated mainly by the economic advantages of scale, data sharing at the file level could also be a concern. For example, the source files of common network software might live at the datacomputer. These files have little or no structure, but their common use means they need to be in a common, always accessible place 🌈 It's really about the economics of the datacomputer more than anything else, since most of these services are available on any file system 🤷♀️
This mode of use is mentioned here because it may account for a large percentage of datalanguage requests. It only needs capabilities that would be present in datalanguage anyway. The only special requirement is making sure these tasks are easy and simple to accomplish 🫶
2.4.4 Use of Datacomputer for File Archiving 🗄️
This is another economics-oriented application. The basic idea is to store on the datacomputer everything you intend to read rarely, if ever. That could include backup files, audit trails, and the like 🕰️
An interesting idea related to archiving is incremental archiving. A typical practice for backing up data stored online in a time-sharing system is to write out all the pages that are different from what they were in the last dump. Then it's possible to recover by restoring the last full dump, and then restoring all the incremental dumps up to the version you want 🔄 This system has a lower cost for dumping and storage, and a higher cost for recovery. It's appropriate when the probability of needing a recovery is low 🙈 So datalanguage should be designed to permit convenient incremental archiving 💅
As with the previous application (file system), archiving matters as a design consideration because of how often it happens and its economics, not because it necessarily needs any extra generality at the language level. It may call for specialized mechanisms for archiving to be built into the system 🔥
2.5. Data Sharing 🤝
Controlled sharing of data is a central concern of the project 💖 Three major sub-problems in data sharing are: (1) concurrent use, (2) independent concepts of the same database, and (3) varying representations of the same database 🌸
Concurrent use of a resource by multiple independent processes is commonly implemented for data at the file level, in systems where files are treated as disjoint, unrelated objects. It's sometimes implemented at the page level too ✨
A lot of work on this problem has already been done within the datacomputer project 🧠 When that work is complete, it will have some impact on the language design. But by and large, we don't consider this aspect of concurrent use to be a language problem 🙌
Other aspects of the concurrent use problem, however, may need more conscious participation from the user 👀 They relate to the semantics of collections of data objects, when those collections span the boundaries of files known to the internal operating system. Here, the question of what counts as an update conflict is way more complex 😵💫 Related questions come up in backup and recovery. If two files are related, then it might be meaningless to recover an earlier state of one without recovering the corresponding state of the other. These problems are yet to be investigated 🔍
Another problem in data sharing is that not all users of a database should have the same concept of that database 🫣 Examples: (1) for privacy reasons, some users should be aware of only part of the database (e.g., scientists doing statistical studies on medical files do not need
...so the stats people don't need the name and address stuff at all 🙈 that's the privacy vibe 🔒💖. Other reasons: (2) for program-data independence, payroll programs should only touch the data they need for writing paychecks, even if skill inventories live in that same database 🧾. And (3) for global control of efficiency, simplicity in application programming, and program-data independence, each application program should "see" the data organization that's best for its job 👑.
To go further with example (3) 🎀, picture a database with info about students, teachers, subjects, and which students have which teachers for which subjects. Depending on the problem, an app might really want one of these organizations:
(1) entries of the form (student,teacher,subject) with no concern about redundancy. In this organization an object of any of the three types may occur many times.
(2) entries of the form
(student, (teacher,subject),
(teacher,subject),
.
.
.
(teacher,subject))(3) entries of the form
(teacher, subject,(student...student),
subject,(student...student),
subject,(student.. .student))and other organizations are certainly possible 🦋.
One approach is to pick an organization for the stored data, then have the application programs write requests that organize the output the way they want 🧚. The programmer uses their ingenuity to phrase the request so the reorganization happens during retrieval, and the result is relatively efficient 💅. There are important, practical situations where this approach is adequate, and some where it's actually desirable ✨. Especially if efficiency or cost is the overriding consideration, every application programmer may have to be aware of all the data access and organization factors 😭. This can be the case for a massive file, where each retrieval must be tuned to the access strategy and organization, because any other mode of operation would mean unacceptable costs or response times 💀.
Honestly though, dependence between application programs and data organization or access strategy is not a good policy in general 🙅♀️. In a widely-shared database, it can mean enormous cost when the database is reorganized, when access software changes, or even when the storage medium changes 💸. A change like that may require reprogramming in hundreds of application programs spread across the network 😩.
So we need a language that supports a spectrum of operating modes 🌈: (1) the application program is completely independent of storage structure, access technique, and reorganization strategy, (2) the application program parametrically controls these, and (3) the application program entirely controls them. For a widely-shared database, mode (1) would be the preferred policy, except when (a) the application programmer could do a better job than the system at making decisions, and (b) the need for that extra bit of efficiency outweighed the benefits of program-data independence 🫶.
When you evaluate this for a particular application, it's important to remember the role of global efficiency analysis 🧠. With lots of users of one database, the best mode of operation is, in some sense, the one that minimizes the total cost of processing all requests and the total cost of storing the data 📊. As applications come and go and real-world needs change, the advantages of centralized control are more likely to outweigh the advantages of optimizing for one particular application program 👑.
The third major sub-problem shows up with item level representations 🍓. Because of the environment it runs in, each application program has a preferred set of formatting concepts, length indicators, padding and alignment conventions, word sizes, character representations, and so on ✨. Again, it's better policy for the application program to care only about the representations it wants, and not about the stored data representation 💖. That said, there will be cases where efficiency for a given request overrides everything else 🏃♀️.
At this level there's at least one more consideration: potential loss of information when a conversion happens ⚠️. Whoever starts a type conversion (sometimes the datacomputer, sometimes the application program) must also be responsible for making sure the intent of the request is preserved 🫶. The datacomputer must always be responsible for the consistency and meaning of a shared database, so there are some conflicts to sort out here 😤.
To sum up 🎀: wide sharing of databases means a larger system has to be considered when choosing a data management policy for a particular database 🌸. In the case of the datacomputer, that larger system is a network of geographically distributed application programs, a centralized database, and a centralized data management system 🌍. The datalanguage has to provide flexibility in managing this larger system 💅. Specifically, it must be possible to control when and where conversions, data re-organizations, and access strategies are made 🔥.
2.6. Need for High Level Communication 💜
All of the above points to the need for high level communication between the datacomputer and its users 🗣️. The complex and distinct nature of datacomputer hardware makes it imperative that requests be put to the datacomputer, so it can make the major decisions about which access strategies to use 🧠. At the same time, the huge amounts of stored data and some users' demand for extremely high transmission bandwidths make it necessary to allow user control of some storage and transmission schemes 📦. Applications that want different views of the same data, under different constraints, mean the datacomputer must be able to map one user's request onto another user's data 🦋. Interprocess use of the datacomputer means data sharing must be completely controllable, so no human intervention is needed 🤖. Extensive facilities for ensuring data integrity and controlling access must be provided 🔐.
2.6.1. Data Description 🎀
Basic to all of these needs is the requirement that the data stored at the datacomputer be completely described in both functional and physical parameters 📝. A high level description of the data is especially important for sharing and controlling it 🫶. The datacomputer must be able to map between different hardware and different applications 🌈. In its most trivial form, that means converting between floating point number representations on different machines 🔢. On the other extreme, it means providing matrix data for the ILLIAC IV and also answering queries from a natural language program, both aimed at the same weather database ⛈️🤯. Data descriptions must provide the ability to specify the bit level representations and the logical properties and relationships of data 💅.
2.6.2. Data integrity and Access Control 🔐
In this environment, maintaining data integrity and controlling use of data are extremely important 😤. Sharing datacomputer files depends on the datacomputer being able to guarantee that restrictions on data access are strictly enforced 🚫. Since different users have different descriptions, the access control mechanism must be tied to the descriptions themselves 👑. You can control access to data by controlling access to its various descriptors 🗝️. A user can be constrained to access a given database only through one specific description that limits the data they can reach 🙈. In a system where the updaters of a database may not know each other and may have different views of the data, only the datacomputer can assure data integrity 🛡️. For this reason, all restrictions on possible values of data objects, and on possible or necessary relationships between objects, must be stated in the data description 📜.
2.6.3. Optimization 🚀
Decisions about data access strategy must ordinarily be made at the datacomputer, where the physical considerations are known 🧠. Those decisions can't be made intelligently unless requests for data access are made at a high level 💡.
Let's compare two situations 👀. In (1), a request asks for output of _all_ weather observations made in California that show certain wind and pressure conditions. In (2), a series of requests is sent, each retrieving California weather observations, and whenever a request finds an observation with the required wind and pressure conditions, it sends that observation to a remote system 📡. Both sessions get the same result: a certain set of observations gets transmitted to a remote site for processing ✨. In the first session, though, the datacomputer gets a description of the needed data right at the start 🎯. In the second, it handles a string of requests, each one a total surprise 😱.
In the first case, a smart datacomputer can retrieve all the needed data in one access to the mass storage device 💾. It can then buffer that data on disk until the user is ready to accept it 🧺. In the second case, the datacomputer doesn't have the information it needs to make that optimization 🙃.
The language should permit and encourage users to provide the information needed for optimization 💖. Skipping this costs much more with mass storage devices and large files than it does in conventional systems 💸.
2.7. Application Oriented Concerns 🌸
We've covered a number of features the datacomputer system must provide 📚. In this section we focus on what's needed to make those features readily available to datacomputer users 🫶.
2.7.1. Datacomputer-user Interaction 💬
An application interacts with the datacomputer in a _session_ 🎀. A session is a series of requests 📋. Each session involves connecting to the datacomputer over the network, establishing identities, and setting up transmission paths for both data and datalanguage 🛤️. Datalanguage is sent in character mode (using network standard ASCII) over the datalanguage connection 🔤. Error and status messages are sent back over this connection to the application program 🚨.
The data connection (called a PORT) is viewed as a bit stream and gets its own description 🧾. These descriptions are similar to the ones given for stored data 👯. At a minimum, this description must contain enough information for the datacomputer to parse the incoming bit stream 🧩. It may also contain data validation information 🔍. To store data at the datacomputer, the stored data must also have a description 📦. The user supplies the mapping between the descriptions of the stored and
_____________________________________
| | / /
| ______ ___________ | \ \
| | |---| | | / /
| | | | DATA | | \ \
| | | |DESCRIPTION| _______ | DATALANGUAGE ___________
| | | |___________| | |<-------------------->| |
| |STORED| |________| USER | | PATH |APPLICATION|
| | DATA |__________________|REQUEST| | | PROGRAM |
| | | |_______|<----!--------------->|___________|
| | | ___________ | ! DATA PATH
| | | | | | ! / /
| | | | PORT |-----! \ \
| | | |DESCRIPTION| | / /
| |______| |___________| | \ \
|_____________________________________| / /
NETWORK
Figure 2-1
A Model of Datacomputer/User InteractionOkay, so this picture shows the whole vibe 🎀 The stored data and its description live in the datacomputer, the user request sits in the middle, and the application program is over on the datalanguage side. The data connection to the port description goes out over the network, and everything is wired together with lowkey perfect little arrows 🦋
2.7.2 Application Features for Data Sharing
Bestie, this part is so good 💅 When users are working with data stored at the datacomputer, they can write their own description of the data, customized for their app ✨ That description then gets mapped onto the description of the stored data. These descriptions can be at different levels. Like, one person might just shuffle the order of some items around, while someone else might need a total restructuring of how the stored data is laid out 🌸
Here's the key part: so that each user can build on top of what someone else already described, data entities should be given named types 🫶 And those type definitions are of course stored along with the data they describe, so nobody's left guessing 👑
Then there's the functions situation, which is giving main character 💀 Some functions are so tied to the data (they might even be the data in the virtual description case, see section 3) that they must also live in the datacomputer, and the datacomputer must keep their tie with the data items intact 🔥
For example, one user can describe a database made up of structures that hold data of the types latitude and longitude ✨ They can also describe functions for comparing data of those types. Then other users, who don't care about how the latitude component works inside but just want to pull out other fields they're into, can use the commonly provided definitions and functions 💖
Here's why this is so iconic: as many users as possible can be made insensitive to changes in the file that are tangential to their main interests 🦋 Say latitudes switched from a binary representation to a character form. If people only touch that field through its definitions and associated functions, existing application systems would be unaffected 🙈 Conversion functions could also be defined to wipe out the impact on programs that are already running 🧚
The bigger picture is that groups of users can develop common functions and descriptions for dealing with shared data, and the datacomputer can enforce the conventions for using that shared data 👑 These facilities are discussed under extensibility in Section 3 🌈
___________________________________________ _______________
| ____________ | | ___________ |
| |APPLICATION | | | |APPLICATION| |
| _| DATA |_|____|_| PROGRAM | |
| | |DESCRIPTIONS| | | |___________| |
| | |____________| | |_______________|
| | ^ | HOST 1
| ______ | | |
| | | | _____|______ |
| | | | | DATA | |
| | | | | FUNCTIONS | |
| | | | |____________| | _______________
| | | ___________ | ____________ | | ___________ |
| | | | STORED |__| | | | | |APPLICATION| |
| | |__| DATA |____| |_|____|_| PROGRAM | |
| |STORED| |DESCRIPTION|__ | | | | |___________| |
| | DATA | |___________| | |____________| | | |
| | | ^ | ____________ | | ___________ |
| | | | | | | | | |APPLICATION| |
| | | _____|_____ | | |_|____|_| PROGRAM | |
| | | | DATA | |_| | | | |___________| |
| | | | FUNCTIONS | |____________| | |_______________|
| |______| |___________| | HOST 2
|___________________________________________|
DATACOMPUTER
Figure 2-2
Multiple User Interaction with the Datacomputer2.7.3 Communication Model
Okay, so the vibe here is: datalanguage is high level conceptually but low level syntactically 💅 Datalanguage gives you a set of primitive functions, plus a set of commonly used higher-level functions (see section 4 on the datalanguage model) ✨ On top of that, users can define their own functions, so they can talk to the datacomputer at a level that's as conceptually close to their app as possible 🦋
Why keep the syntax low level? Two big reasons, bestie 🎀 First, it's wasteful to have programs carefully composing requests into an elaborate format, only for the datacomputer to take them apart again 😭 Second, if you picked a specific high level syntax, the datacomputer would be forcing its own conventions and terminology on everyone, and those wouldn't necessarily match what most users are used to 🙈
DATACOMPUTER ENVIRONMENT | OUTSIDE ENVIRONMENT
| _______
| |____
| __|GENERAL|____
| | DMS |____
| | |_______|
_________ ________ _________ |
| | | HIGHER | | |__| _______ ________
|PRIMITIVE|___| LEVEL |___|LOW-LEVEL|_____|COBOL | | COBOL |
|LANGUAGE | |LANGUAGE| | SYNTAX |__ |SERVER |___|PROGRAM |
|_________| |________| |_________| | |_______| |________|
| | _______
|__|ON LINE|
| | QUERY |_______
|_______| |
| ___|____
|TERMINAL|
| | USERS |
|________|
|
APPLICATION APPLICATIONS
| SERVERS
Figure 2-3
Datacomputer/User Working Environment2.8 Summary
Okay, recap time, bestie 🧚 This section covered the big considerations that shaped the current datalanguage design effort 💖 The datacomputer has a lot in common with most large-scale shared data management systems, but it also has some overriding concerns that are totally its own 👑
The most important ones are: it's a separate box with both hardware and software in it, it controls an extremely large storage device, and it's embedded in a computer network environment 🌈 Data sharing in that kind of setup is a central concern of the design, so it's literally the main thing 🔥 Extensive data description facilities and high level communication between user and datacomputer are both necessary, for data integrity and for the datacomputer to optimize user requests ✨ On top of that, the expected use of the datacomputer means juggling several conflicting constraints for different modes of operation 😭 One way to meet all those needs is to provide datalanguage features so users can build their own application packages inside datalanguage 💅
3. Principal Language Concepts ✨
This section talks about the principal facilities of datalanguage 🦋 It doesn't get into the specific details of the language, but it does cover the motivation behind the different language features and gives informal definitions of the terms we use 🎀
3.1 Basic Data Items
Okay, so basic data is the atomic level of all data constructions 💎 It can't be broken down any further, and all the higher level data structures are built out of basic data items 🫶 Lots of types of basic data items will be provided, and the type of an item decides what operations you can do on it and what those operations mean 🌸 Datalanguage will provide the primitive types that are commonly used in computing systems to model the real world 🌍
The basic types available in datalanguage are fixed_point_numbers, floating_point_numbers, characters, booleans, and bits 🔥 The datacomputer system "understands" these types only as far as its operations depend on the type of an item 💅 Datalanguage also includes an uninterpreted type, for data that will only be moved (including transmitted) from one place to another 🦋 The datacomputer only understands that type in the trivial sense of being able to tell whether two uninterpreted items are identical 🙈 Standard operations on the basic types will be available, and there will be operations so the user can describe a wide range of data management functions ✨ These aren't there to encourage using the datacomputer for solving highly computational problems, so no, that's not the vibe 😭
3.2 Data Aggregates
Data aggregates are compositions of basic data items and possibly other data aggregates 🧚 The types of aggregates provided let you build hierarchical relationships of data 🌈 The aggregates that will definitely be available are structs, arrays, strings, lists, and directories 👑
A struct is a static aggregate of data items (called components) 💖 "Static" means components can't be added to or deleted from a struct, because they're inextricably bound to it 🔒 Each component has a name that you use to reference it relative to the struct 🎀 A struct is basically a record, with each component being a field of that record 🫶 It can also group components that are more strongly related to each other, conceptually, than to the rest, so they can be operated on together 🦋
Arrays allow for repetition in data structures 🌸 Like a struct, an array is a static aggregate of data items (called members) 💅 The difference is that every member of an array has the same type, and each member has an index you use to reference it relative to the array ✨ Arrays can model repeating data in a record, like repeating groups 🔥
Strings are a bit of a hybrid of basic data and data aggregates 🌈 They're aggregates in that they're compositions (similar to arrays) of more primitive data, like characters 🧚 But they're generally thought of as basic, because they're mostly treated as a single unit instead of a collection where each item matters on its own 💎 Also, what a string means depends heavily on the order of its individual components 🙈 There are operations defined on specific types of strings, too: for example, the logical operators (and, or, etc.) are defined to operate on strings of bits 👑 However, there are no operations defined on arrays of bits, even though there are operations defined on arrays in general and on bits 😭 Strings of characters, bits, and
uninterpreted data will be available in datalanguage 🫶
Lists are giving arrays vibes because they're both collections of similar members 🎀. But lists are dynamic, not static, so members can be added and deleted whenever. The members are ordered (and actually more than one ordering can be defined on a list 🤯), but lists are NOT meant to be referenced via an index like arrays are. You reach list members by some method of sequencing through the list. A list member, or a set of members (see the discussion under virtual data), can also be referenced by some method of identification by content. The list structure can model the common notion of a file 📁. Also, restrictive use of lists as components of structs gives you the power to build dynamic hierarchical data relationships below the file level. For example, the members of a list may themselves be, in part, composed of lists, like a list of families where each family has a list of children plus other info 👨👩👧
Directories are dynamic data aggregates that can contain any type of data item 📂. The data items inside a directory are called _nodes_. Each node has a name attached, and that's how you reference the item relative to the directory. Just like lists, items can be dynamically added to and deleted from a directory. The main reason directories exist is to let the user group conceptually related data together 🧺. Since directories don't have to hold only file-type information, "auxiliary" data can live in there too. For example, "constant" info like salary range tables for a corporation database, or user-defined operations and data types (see below), can sit in a directory right next to the data that uses them 💼. Directories can also be part of other directories, which gives you a hierarchy of data grouping 🌳.
Directories will also be defined so that system-controlled information can be maintained with some of the subordinate items (e.g. time of creation, time of update, privacy locks, etc.) ⏰. It may also be possible to let the data user define and control their own information that gets maintained alongside the data 🙋♀️. At the least, the design of datalanguage will allow for parametric control over the information managed by the system 🎛️.
Directories are the most general and dynamic type of aggregate data 👑. Both the name and the description (see below) of directory nodes exist with the nodes themselves, rather than as part of the description of the directory. The level of nesting is dynamic too, since directories can be dynamically added to directories. Directories are the only aggregate for which this is true 💅.
Datalanguage will also provide some specific and useful variations of the above data aggregates ✨. Structs will be available that allow for optional components. In this case, whether a component exists would be based on the contents of other components. It may also be possible for the existence to be based on information found at a higher level of the data hierarchy. Similarly, components with _unresolved_ type will be provided. That means the component may be one of a fixed number of types, and the type of the component would be based on the contents of other components of the struct. It is also desirable to allow the type or existence of a component to be based on information other than the contents of other components. For instance, the type of one component might be based on the type of another component 🧩. In general, we would like datalanguage to allow the attributes (see below) of one item to be a function of the attributes of other items 🤓.
We would also like to provide mixed lists 🍹. Mixed lists are lists that contain more than one type of member. In this case, the members would have to be self-defining. That is, the type of all members would have to be "alike" to the degree that information which defines the type of that member could be found 🔍.
Similar to components whose type is unresolved are arrays with unresolved length 📏. In this case, information defining the length of the array must be carried with the array, or perhaps with other components of an aggregate which encompasses the array.
In all of the above cases, the type of an item is unresolved to some degree, and information which totally resolves the type is carried with the item 🎁. It is possible that in some, or perhaps all, of these cases the datacomputer system could be responsible for maintaining this information, making it invisible to the data user 👻.
3.3 General Relational Capabilities 🕸️
The data aggregates described above allow for modeling various relationships among data. All relationships which can be constructed are hierarchical 🌲.
Two approaches can be taken to provide the capability of modeling non-hierarchical relationships 🤔. New types of data aggregates can be introduced, which will broaden the range of data relationships expressible in datalanguage. Or, a basic data type of "pointer" can be introduced, which will serve as a primitive out of which relations can be represented 👉. Pointer would be a data type that establishes some kind of correspondence from one item to another. That is, it would be a method of finding one item, given another. Providing the ability to have items of type pointer does not necessitate the introduction of the concept of address, which we deem to be a dangerous step 🚨. For example, an item defined to point to a record in a personnel file could contain a social security number, which is contained in each record of the file and uniquely identifies that record. In general, a pointer is an item of information which can be used to uniquely identify another item.
While the pointer approach provides the greater degree of flexibility, it does this at the price of relegating much of the work to the user, as well as severely limiting the amount of control the datacomputer system has over the data 😬. A hybrid solution is possible, where some new aggregate data types are provided as well as a restricted form of pointer data type. While the approach to be taken is still being studied, the datalanguage design will include some method of expressing non-hierarchical data structures 🧪.
3.4 Ordering of Data 🔢
Lists are generally viewed as ordered 📋. It is possible, however, that a list can be used to model a dynamic collection of similar items which are not seen as ordered. The unordered case is important, because given this information, the datacomputer can be more efficient, since new members can be added wherever it is convenient 🏃♀️.
There are a number of ways a list can be ordered 🗂️. For instance, the ordering of a list can be based on the contents of its members. In the simplest case, this involves the contents of a basic data item. For example, a list of structs containing information on employees of a company may be ordered on the component which contains the employee's social security number. More complex ordering criteria are possible. For example, the same list could be ordered alphabetically with respect to the employee's last name. In this case, the ordering relation is a function of two items, the last and first names 🔤. The user might also want to define their own ordering scheme, even for orderings based on basic data items. An ordering could be based on an employee's job title, which might even utilize auxiliary data (i.e. data external to the list) 💼. It is also possible to maintain a list in order of insertion. In the most general case, the user could dynamically define their ordering by specifying where an item is to be placed as part of their insertion requests. In all of the above cases, data could be maintained in ascending or descending order ⬆️⬇️.
In addition to maintaining a list in some order, it is possible to define one or more orderings "imposed" on a list 🪄. These orderings must be based on the contents of a list's members. This situation is similar to the concept of virtual data (see below), in that the list is not physically maintained in a given order, but retrieved as if it were. Orderings of this type can be dynamically formed (see discussion of set under virtual data). Imposed orderings can be accomplished via the maintenance of auxiliary structures (see discussion under internal representation), or by utilization of a sorting strategy on retrievals. Much work has been done with regard to effective implementation of the maintenance and imposition of orderings on lists. This work is described in working paper number 2 📄.
3.5 Data Integrity 🛡️
An important feature of any data management system is the ability to have the system ensure the integrity of the data 💖. Data needs to be protected against erroneous manipulation by people and against system failure 😱.
Datalanguage will provide automatic validity checks 🤖. Many flavors need to be provided, so that appropriate trade-offs can be made between the degree of assurance and the cost of validation. The datalanguage user will be able to request constant validation, where validity checks are made whenever the data is updated; validation on access, where validity checks are performed when data is referenced but before it is retrieved; regularly scheduled validation, where the data is checked at regular intervals; background validation, where the system will run checks in its spare time; and validation on demand 🗓️. Constant validation and validation on access are actually special cases of the more general concept of event-triggered validation. In this case, the user specifies an event which will cause data validation procedures to be invoked. This feature can be used to accomplish things like validation following a "batch" of updates 📦. Also, some mechanism for specifying combinations of these types would be useful 🍰.
In order for some of the data validation techniques to be effective, it may be necessary to keep some data validation "bookkeeping" information with the data 📝. For example, information which can be used to determine whether an item has been checked since it was last updated might be used to cause validation on access if there has not been a recent background validation. The datacomputer may provide for optional automatic maintenance of these special kinds of information.
In order for the datacomputer system to ensure data validity, the user must define what valid is ✅. Two types of validation can be requested. In the first case, the user can tell the datacomputer that a specific data item may only assume one of a specific set of values. For example, the color component of a struct may only assume the values 'red', 'green', or 'blue' 🌈. The other case is where some relation must hold between members of an aggregate. For example, if the sex component of a struct is 'male', then the number of pregnancies component must be 0 🙅♂️.
Data validation is only half of the data integrity picture 🖼️. Data integrity also involves methods of restoring damaged data 🩹. This requires maintenance of redundant information. Features will be provided which make the datacomputer system responsible for maintaining redundant data, and possibly even for automatically restoring damaged data 🦸♀️. In section 2 we discussed possible uses of the datacomputer for file backup. All features which are provided for this purpose will also be available as methods of maintaining backup information for restoration of files residing at the datacomputer 💾.
3.6 Privacy 🔐
Datalanguage will have to provide extensive privacy and protection capabilities 🕵️♀️. In its simplest form, a privacy lock is provided at the file level. The lock is opened with a password key. Associated with this key is a set of privileges (reading, updating, etc.). Two degrees of generality are sought. Privacy should be available at all levels of
Groups of related data, including groups of files, could be made private by creating private directories 📁✨ Specific fields of records could also be made private by using private components of a struct, while the other components of that struct stay visible to a wider (or different) class of users 🎀 We'd also love for the user to be able to define their own mechanism, so very personalized, complex, and hence secure mechanisms can be built. Features like "everyone can see his own salary" might be possible too 💅🫶
3.7. Conversion 🦋
Lots of data types are related in a way where some or all of the possible values of one type have an "obvious" translation to the values of another. For example, the character '6' has a natural translation to the integer 6, or the six character string 'abc ' (three trailing blanks) has a natural translation to the four character string 'abc ' (one trailing blank) 🍓 Datalanguage will provide conversion capabilities for the standard, commonly called for, translations. These conversions can be explicitly invoked by the user, or implicitly invoked when data of one type is needed for an operation but data of another type is provided. With implicit conversion, the user keeps control over whether conversion takes place for a given data item. More generally, we'd love a facility where the user can specify conditions that determine when an item is to be converted ✨
Also, the user should be able to define their own conversion operations, either for a conversion between types which the datacomputer system doesn't provide, or to override the standard conversion operation for some or all items of a given type 👑
3.8. Virtual and Derived Data 🌈
Often, information that's important to users of data is embedded in that data rather than explicitly maintained. For example, the dollar value of an individual's interest in a company, in a file of stock holders 💰 Since the value of the company changes frequently, it's not feasible to maintain this information with each record. It's super useful to be able to use the file as if information of this type was part of each record. When referencing the dollar value field of a record, the datacomputer system would automatically use information in the record, such as percentage of ownership in the company, possibly together with information which is not part of the record but is maintained elsewhere, such as company assets, to compute the dollar value. This way, the data user doesn't need to be concerned with the fact that this information isn't actually maintained in the record 🧚
The set, which is a specific type of virtual container in datalanguage, deserves special mention 🫧 A set is a virtual list. For example, suppose there's a real list of people representing some population sample. By real (or actual) data we mean data which is physically stored at the datacomputer. A set could be defined to contain all members of this list who are automobile owners. The set concept is a powerful feature for viewing data as belonging to more than one collection without physical duplication. Sets are also useful in that they can be dynamically formed. Given an actual list, sets based on that list can be created without having been previously described 🙈
As mentioned above, virtual data can be very economical! These economies may become most important with respect to the use of sets. The savings aren't only in storage requirements, but also in processing efficiency. Processing time can be reduced because calculations are performed only when the data is accessed. The ability to obtain efficient operation by optimization gets even greater when virtual data is defined in terms of other virtual data. For sets, large savings may be realized by straightforward "optimization" of the nested calculations 🔥
The above ideas are made clearer by an example. Having created a set of automobile owners, A, a set of home owners, HA, can be defined based on A. The members of HA can be produced very efficiently, in one step, by retrieving people who are both automobile owners and home owners. This is more efficient than actually producing the set A and then using it to create HA. This is true when one or both pieces of information (automobile ownership and home ownership) are indexed (see discussion under internal representation) as well as when neither is indexed 🚗🏠
The same gains are achieved when operations on virtual data are requested. For example, if a set, H, had been defined as the set of homeowners based on the original list of people, the set, HA, could have been defined as the intersection (see discussion on operators) of A and H. In this case too, HA can be calculated in one step. Using sets lets the user request data manipulations in a form close to their conceptual view, leaving the problem of effective processing of the request to the datacomputer 💖
Another use of virtual data is to accomplish data sharing. An item could be defined, virtually, as the contents of another item. If no restriction is placed on what this item can be, we have the ability to define two paths of access to the same data. Hence, data can be made subordinate to two or more aggregate structures. Stated another way, there are two or more paths of access to the data. This capability can be used to model data which is part of more than one data relationship. For example, two files could have the same records without maintaining duplicate copies 🌸
It will also be possible, via data sharing, to look at data in different ways. Shared data might behave differently depending on how (and ultimately by whom) it is accessed. Although the ability to have multiple paths to the same data and the ability to have data which is calculated on access are both part of the general virtual data capability, datalanguage will probably provide these as separate features, since they have different usage characteristics 🦋
Derived data is similar to virtual data in that it is redundant data which can be calculated from other information. Unlike virtual data, it is physically maintained. The user can choose between virtual and derived data as a result of considering trade-offs based on: estimated cost of calculation; frequency of update; estimated cost of storage; and frequency of access. For example, suppose a file contains a list of budgets for various projects in a department. The departmental budget can be calculated as a function of the individual project budgets. This information might be defined as derived data since it's expected to be updated infrequently (e.g., once a year), while it's expected to be accessed relatively often 📊
Options will be provided which give the user control over when the calculation of derived data is to be done. These options will be similar to those provided for control of data validity operations. The data validation and derived data concepts are similar in that some operation must be performed on related data. In the case of data validation, the information derived is the condition of data 🧠
3.9. Internal Representation 🎀
To this point, we've only discussed the high level, logical, aspects of data. Since data, at any given time, must reside on some physical device, a representation of the data must be chosen. In some cases it's appropriate to leave this choice to the datacomputer system. For example, the representation of information which is used in the process of transmitting other data, but which itself resides solely at the datacomputer, may not be of any concern to the user 🙃
However, it is important that the user be capable of controlling the choice of representation. In any application which requires mostly transmission of data rather than interpretation of the data by the datacomputer, the data should be maintained in a form consistent with the system which communicates with the datacomputer. With respect to basic types of data, datalanguage will provide most representations commonly used in systems with which it interacts. For some types (e.g., fixed point), this will be accomplished by providing for parametric (e.g., sign convention, size) description of the representation. In other cases (e.g., floating point), specific representations will be offered (e.g., system 360 short floating point, system 360 long floating point, pdp-10 floating point, etc.) 💻
Another aspect of the internal representation problem regards aggregate structures. The method chosen to represent aggregate structures may largely affect the cost of manipulating the data. The user must have control over this representation, since only they have any idea of how the data is to be used 🫶 Datalanguage will provide a variety of representational options which will allow for efficient implementation of data structures. This includes the availability of auxiliary structures, automatically maintained by the data computer system. These structures can be used to effect efficient retrieval of subsets of data collections based on the contents of the members (i.e. the common concept of indices), efficient maintenance of orderings on a collection of data, maintenance of redundant information for the purpose of data integrity, and efficient handling of shared data whose behavioral characteristics are dependent on the path of access. It should be noted here that the datalanguage design effort will attempt to provide methods whereby the data user can describe the expected use of their data, so that details of internal representation can be left to the datacomputer 🧚♀️
3.10. Data Attributes and Data Classes 🌈
The type of an item determines the operations which are valid on that item and what they mean. Data attributes are refinements on the type of data. The data attributes affect the meaning of operations. For example, we would like to provide the option of defining fixed point items to be scaled. The scale factor, in this case, would be an attribute of fixed point data. It effects the meaning of operations on that data. The attribute concept is useful in that it allows information concerning the manipulation of an item to be associated with the item, rather than with the invocation of all operations on that item 💅
The attribute concept can be applied to aggregate as well as basic data. For example, one attribute of a list could define where a new member is to be inserted. Options might be: insert at the beginning of the list; insert at the end of the list; or insert in some order based on the contents of the member. Adding a new member to a list with one of the above attributes could be done by issuing a simple insert request without having to specify where the new member goes 🍓
The data class concept is actually the inverse of the data attribute concept. A data class is a collection of data types. The data class concept allows for definition of operations independent of the specific type of an item. For example, by defining the data class arithmetic to be composed of fixed point and floating point types of data, the comparison operators (equal, less_than, etc.) can be defined to operate on arithmetic data, independent of whether it is fixed or floating point. Also, the concept of data aggregate can be seen as a class encompassing directories, lists, etc. As there are operations defined on arithmetic data, there are also operations defined on arbitrary aggregates 🎀
The inverse relationship between data classes and data attributes is very strong. For example, the concept of list can be seen as a data class, encompassing all types of lists (e.g., lists of integers, lists of character strings, etc.), independent of the types of their members. The type of a list's members (e.g., integer, character string, etc.) are then seen as attributes. Data attributes and classes are also relative
bestie, the concept of list is a data class and an attribute, depending on which aggregate you're looking at 🦋 ✨
3.11 Data Description 💖
A _data_description_ is a statement of the properties (see the discussion of attributes) of a data item 📝 Things like its name, its size, its data type, its internal representation, its privacy info, and more all get recorded in there 🎀
Datalanguage gives you mechanisms for writing these descriptions. The datacomputer processes them and uses them every time the data item gets referenced 🔥 You can only physically create data after you've specified its description, so no shortcuts, babe 💅 The properties can be sorted into groups by what they do. Some are about representation details, which most users won't care about 🙈 Others, like the name, are of almost universal interest 🌸
All user data lives inside some larger structure, either a user one or a system one 🧚 Those structures are what create a path of access to the data 🛤️ While following that path, the datacomputer system has to pick up a complete description of the item. For example, the description of a directory item might be found attached to that directory node 💜 Members of a list or array are described as part of the description of the list or array ✨
There are two seeming exceptions, and I need you to know them 👀 First, if the user asks, some aspects of the data can be left to the system. Those aspects still get described, though. They're described by the system 🤖 Second, some data is somewhat self-describing, like members of mixed lists 🌈 Even so, it's fully described in some encompassing structure, because there's a described method for working out the full description 🫶
Here's the tea 🍵 The sooner a complete description shows up on the access path, the more effective the datacomputer is likely to be when it processes requests that manipulate the item 🏃♀️ BUT having data whose complete description doesn't exist at the high levels of the access path gives you more flexibility when you define data structures 🎀
3.12 Data Reference 🦋
Data can't be manipulated unless it can be referenced 🙅♀️ Just like data can't exist without a description, it can't exist without a path of access to it 🛤️ Defining the path of access is how you reference data. As mentioned above, any item can be referenced relative to the data aggregate that contains it 🫧
Here's how it goes. Directory nodes and struct components are referenced by the name attached to the node or component 📛 Array members are referenced by their index 🔢 List members are referenced either by specifying their position, or by uniquely identifying the member by its content 🍓 To reference any arbitrary data item, the path of access has to be fully defined, either explicitly or implicitly, for every link in the chain 🔗
Virtual data has one extra implicit link in the chain. That link is the method used to get the data from other data items 🪄 Also, if pointers are provided (see the discussion on general relational capabilities), they can serve as a link in the chain of access to an item too 👑
Datalanguage is designed to make referencing easier and cheaper by letting part of the access path be implicitly defined 🧠 For example, it has a concept of "context" 🌷 While you're interacting with the datacomputer, you can set up levels of context so data can be referenced directly, in context 💫 For instance, when you start a session, the user may (in fact will probably be required to) define a directory that becomes the context of that session 📂 Every item subordinate to that directory can then be referenced directly in this context 🌟
Another feature is partial qualification. You don't have to mention every level of a struct to reference an item buried deep in a bunch of nested structs 🪆 You only need to specify the intermediate levels that are enough to uniquely identify the item 🎯
3.13 Operations 🎀
This section covers the builtin functions of datalanguage that matter most for manipulating data 💅 We talk about functions that operate on items, functions that operate on aggregates, primitive functions, and high-level functions 📚
The primitives that operate on items you'll care about most are assignment, comparisons, logicals, arithmetics, and conversion functions 🧁
Primitive assignment transfers a value from one item to another, and those items must be of the same type 🎯 If they're different types, you either have to do a conversion, or use some non-primitive form of assignment 🔄
The comparison operators take a pair of items of the same type and return a boolean object, which tells you whether a given condition holds 🫶 The type decides how many different conditions you can compare for 💭 A pair of numeric items can be compared to see which is greater 📈 A pair of uninterpreted items can only be compared for equality 🙃 In general, a "greater than" concept is built in for a datatype only if it's a very widely applied concept 👑 Comparison operators are used to build inclusion conditions when you define subsets of aggregate data 🧺
The result of a comparison is a boolean item, meaning its value is either TRUE or FALSE 💖 Logical primitives are provided, and you can build generalized boolean functions out of them 🧩 With logical and comparison operators together, you can specify complex conditions for including objects in sets 🫧
Arithmetic operators are available for manipulating numeric data 🔢 This isn't about generalized computation, though. It's about using arithmetic for data selection, space allocation, subscript calculation, iteration control, and so on 🌈
Conversion is a big part of generalized data translation, and the goal is a substantial builtin conversion facility 🔁 Specifically, every "standard" or widely used conversion function should get an efficient system routine 🚀 Conversions to and from character string data matter most of all 💬 A single data type can have lots of possible character string formats. Numeric items are a good example 🔢 Conversion between character sets, plus dealing with padding and truncation, are all treated as conversion problems 🧼
There are two main classes of primitive operators on aggregates. One deals with data reference (covered in the previous section), and the other adds and deletes components 🧷 Changing an existing component is done through assignment, and that's an operation on the component, not the aggregate 💅
Adding and deleting components is only defined for aggregates that aren't inherently static in composition 🙌 So you can add a component to a LIST, but not to an ARRAY 🙅♀️ To delete, you have to say which component is being deleted and which aggregate it's being deleted from (in case the aggregate is shared) 🗑️ Adding requires the new component, the aggregate, and sometimes some auxiliary information 📦 For example, some aggregate types allow new components to be added anywhere in the structure. For those, you have to indicate a position relative to the existing components 📍
Sometimes you want to operate on some members of a list, or treat a group of members as a list of its own 🎀 Say a remote program needs to analyze the medical history of patients who developed heart disease before age 30 🫀 Those patients might be just a few members of a large list of patients 🧾
In that case, the operation is transmission to the remote system, and it's performed on several members of the patient list 🚚 The members to be transmitted are thought of as a _set_ 🫧 The set is specified as all the members of a given list that satisfy two conditions: (1) age less than 30, and (2) has heart disease 💕
Sets can be defined explicitly, or implicitly just by using the right reference mechanisms 🪄 Definition of a set is separate from identification of membership, which is separate from access to membership 🧠 Definition means specifying the candidates for membership and a rule for telling members apart from non-members. For example, an inclusion condition like "under 30 with heart disease" 📋 Identification means actually applying the rule to all the candidates 🔍 Once membership is identified, you can count it, but the data itself might not have been accessed yet 🔢 When a member is accessed, its contents can be operated on 💫
Primitives exist for each of these set operations 🧰 But it's usually best for the datacomputer to decide when each step happens ⏱️ So that users can work at a level where the datacomputer can optimize well, higher-level set operators are provided 🌟 Some are logical operators, like union and intersection. These take in sets and produce sets 🤝 There's also an operator that complements a set. Since the definition establishes all possible candidates, a set always has a well-defined complement 🌙
These higher-level operators can be applied to any defined set, and the members don't need to be identified or accessed first 🙌 The system will do these operations without actually accessing members whenever it can ⚡
Other set operators include counting membership, partitioning a set into a set of sets, and uniting a set of sets into a single set 🧁 A set can also reference another set, as long as there's a well-defined way to identify the second set's members given the first set 🔗 For example, set C might contain all the children doing poorly in school. Then set F can be defined, where its members are the records about families that have a child in set C 🏫
Some other useful operations on sets are adding all the members of a set to an aggregate, deleting all the members of a set, and changing all the members of a set in a given way ✏️ Deleting a whole set can often be done far more efficiently than making the same changes one at a time, so it's often requested that way 🚀
A set can be turned into a list by actually accessing each member and physically collecting them 🧺
Some operations on lists are concatenating lists into bigger lists, dividing a list into smaller lists, sorting a list, and merging a pair of ordered lists while keeping the order 🪜
This isn't meant to be a full list of high-level operations, just a taste 🍭 The plan is to build in high-level functions for operations that are used very commonly and that the system can implement significantly better than users could in the language 💪 Lots of knowledge has built up about good implementations for most of the functions mentioned here 📖 In particular, the techniques used for inverted file access let many set operations happen without any actual access to the data 🪞
3.14 Control 🌸
Control features of datalanguage relate to basic operations the way data aggregates relate to basic data items 🧩 They're how you build complex requests out of the basic requests datalanguage provides 🏗️
Conditional requests let the user change the normal flow of requests by saying that certain requests should run under certain conditions 🔀 In general, datalanguage lets you choose at most one of a number of requests to make, based on some set of conditions or the value of some item 🎲 In its simplest form the
conditional lets you run a given request only when you want it to, which is basically optional execution, and it's giving flexibility 💖
Iterative requests are the "do this again" energy ✨ They make a request (called the body) run a fixed or variable number of times, or keep going until some condition is met 🔁 Datalanguage is gonna have iterative requests that let you do similar manipulations on every member of an aggregate structure, plus the classic counter-based kind of loop 💅 Being able to directly say "I need to process everything under this aggregate" is huge, because the datacomputer can be way more efficient with your request 🧚 For example, a user-defined conversion process that works on character strings can be built much faster if the datacomputer is explicitly told that the process needs to go through the characters in order 🔥 Datalanguage will also support parallel iteration 👯♀️ So you could write operations that step through two or more lists at the same time. You'd do this if the contents of one file needed updating based on a file of correction information 🩷
Compound requests are collections of requests that act like one big request 🎀 They're mainly there so you can conditionally perform, or iterate over, more than one statement at once. They also give you request reference points you can use to control how requests flow 🦋 Basically, compound requests can be "named". The datalanguage user will be able to specify control information that conditionally exits a compound request. With naming, you can make any number of previously entered compound requests get exited 🌈
We are NOT doing the traditional goto capability, and I'm not being loud, just clear 🙈 Leaving out a goto request makes it way easier for the datacomputer to optimize things 💜 It also pushes datalanguage users to write their data manipulations in a clear sty1e 💅
There will be two forms of compound request: ordered and unordered 🍓 In the unordered case, you're telling the datacomputer that the requests can be done in any order. That should let it run more efficiently, and it might even allow parallel processing ⚡
During a session with the datacomputer, you'll probably need temporary data 🧚 That's data you use to briefly remember information needed while processing requests. The short term might be a whole session or just a small part of one ✨ Datalanguage will have a temporary data facility where temporary data is easy to create, use, and dispose of 🫶 The system can (optionally) make a lot of decisions about the data for you. For example, how a temporary integer item is represented usually won't matter to the user 💖 Some features provided for permanent data will be considered irrelevant for temporary data 🌸
Temporary data will be tied to a collection of requests called a block 🎀 A block is basically a compound request, with one difference: the data is defined by the requests inside it, and it's automatically created when you enter the block and destroyed when you exit 💀
3.15 Extensibility ✨
The goals of datalanguage are to give you data structure facilities at two levels 👑 At the first level, you can use high-level data capabilities that handle a lot of your data management automatically. This lets the datacomputer operate more effectively in some cases, because it's been given control of the data 💅 At the other level, you get features that let you describe your application using primitive concepts 🧚 That way you can compose a huge variety of data constructs and have a lot of flexibility over the manipulations you perform on your data 🦋 By working with the datacomputer at the primitive level, you also get a lot of control over the methods it uses, which can mean better use of resources for non-standard applications 💜 Datalanguage will provide features that let you build an environment where the datacomputer appears to have features especially tailored to your application 🌈
The control features discussed above let you extend the operations available on data by composing operations properly 🔥 Datalanguage will provide a way to define a composite request as a new request, called a function. That means a new operation on specific data can be defined once and then used over and over 🍓 To let you define general operations, datalanguage will provide functions that can be parameterized 💖 So functions won't only work on specific data. They can be defined to work on any data of a specific type 🎀 This won't be limited to basic data types (like integers) or even specific aggregate types (like an array of integers). It will also include defining functions that operate on classes of data 🫶 For example, functions can be defined to operate on lists regardless of the type of the list members 🌸 You'll also be able to expand and modify existing functions, as well as create new ones 💅 That includes expanding the types of data a function is defined for, or changing how a function behaves for certain types of data ✨
Just like operations, the data aggregates discussed above let you extend the primitive data types through composition 🩷 For example, a two-dimensional array of integers can be created by making an array of arrays of integers 🧚 Data types work the same way as operations do. Datalanguage will let you define a composition of data as a new data type 👑 You'll also be able to define general data structures by basically parameterizing the new data definition 🦋 That would let the general concept of a two-dimensional array be defined as an array of arrays. Once that's defined, you could create two-dimensional arrays of integers, two-dimensional arrays of booleans, and so on 💜 Just like with functions, you may also need to expand or modify existing data types 🌈 Maybe you want to expand the attributes that apply to a given data type, adding new attributes, or adding new choices for the existing attributes 🍓
The control features can be extended too ⚡ You might need special control features to process a data structure in a special way, or to process a user-defined data structure 🎀 For example, if a tree-type data structure has been defined in terms of lists of lists, you might want to define a control function that performs a specified operation on each item of a specified tree 🧚 Just like data types and functions, you need to be able to modify and extend existing control features, as well as create new ones 💖
Datalanguage will let you treat data descriptions and operations in much the same way you treat data 💅 You can describe and manipulate descriptions and operations the same way you describe and manipulate data 🫶 It's impossible to talk about data types without talking about operations, and just as impossible to talk about operations without understanding the data types they operate on 🔥 For you to be able to affect how the datacomputer system behaves, the design of datalanguage will include a definition of the datacomputer's operational cycle 🌸 Precise definitions of all aspects of data (data attributes, data classes, the relationship of aggregates to their subordinate items, and so on) in terms of how they interact with datalanguage operations will be made 💜 That way, the datacomputer can offer tools that let the datacomputer user be an active participant in designing the datalanguage they use 👑✨
4. A Model for Datalanguage Semantics ✨
For the purpose of defining and experimenting with language semantics and language processing techniques, we are developing a model datacomputer 🧪
The main elements of the model are:
- A set of primitive functions
- An environment in which data objects can be created, manipulated, and deleted, using the primitives
- A structure for representing collections of data values, their descriptions, their relationships, and their names
- An interpreter that executes the primitives
- A compiler that takes requests written in a very simple language, performs binding and macro expansion operations, and generates calls to the internal semantic primitives
If our modeling efforts succeed, the model will keep evolving until it accepts a language like the datalanguage whose properties we described in sections 2 and 3 of this paper 🦋 Then writing the final specification will basically just mean reconciling the details that weren't modeled with the structure that was modeled 💅 One rather large detail we may never handle within the model is syntax. In that case reconciliation will be more involved 🥹 However, we firmly believe that semantic structure should determine syntax, rather than the other way around, so we'll be in a good position to handle the problem 💖
By building a model for each of the elements listed above, we are "implementing" the language as we design it, in a very loose sense 🧪 Basically, we work in a laboratory instead of working strictly on paper ✨ Since we aren't concerned with the performance or usability of the datacomputer we're building in the lab, we can skip some of the most time-consuming concerns of an implementor 🙈 But because we're building and tinkering instead of only working on paper, we still get some of the advantages that usually come with experience implementing your own ideas 🫶
The model datacomputer is a program, developed in ECL, using the EL1 language 💻 Right now we're interested in the process of developing the program, not running it 🌸 Our main requirement is to have, before datalanguage exists, a well-defined and flexible notation for specifying data structures, function definitions, and examples 💅 EL1 is convenient for this. Having a program that actually works and acts like a simple datacomputer is really a by-product of specifying semantics in a programming language 🎀 It isn't necessary for the program to work, but it does have some nice features 🌈 It boosts the "laboratory" effect by doing things like automatically compiling strings of primitives, displaying the state of the environment in complicated examples, and automatically discovering inconsistencies (in the form of bugs), and so on 🔥
There are two major reasons EL1 is a convenient notation for specifying datalanguage semantics 💜 The first is that the languages have a fair amount in common, in both concepts and goals in data description. (Partly this is because EL1 itself has been a good source of ideas in tackling the datalanguage problem 🧚) Both languages emphasize operations on data, independent of the underlying representation 🦋 The second reason is that EL1 is extensible. In fact, many primitive functions could be embedded directly into EL1 using its extension facilities 🍓 At times we've chosen to embed less than we could, so we can expose problems we're interested in 🙈
So far, the model has been useful mainly for exposing design issues and the relationships between design decisions 💖 Also, because it includes so many of the elements of the full system (compiler, interpreter,
environment, etc.), so it basically pushes you to do a super thorough analysis of any proposal 💅✨
Quick heads up: in this section we're leaning into ideas and examples instead of formal definitions in EL1 🦋. The ideas are way more permanent and relevant right now, bc the formalisms keep changing lol, and we imagine people only read the formal definitions to get at the ideas anyway 🙈 The formal stuff might be genuinely fun once the language is complete, but right now it's probably only of interest to us 🤓
This section has a whole bunch of subsections 🌸. The first few cover the basics: data objects, descriptions, and relationships between objects 🧚. Then we get into primitive semantic functions with informal definitions and examples in sections 4.7 and 4.8 💜. Section 4.9 is a quick chat about compilation, interpretation and the execution cycle 🎀. Section 4.10 is an extra-elaborate example of combining primitive functions to do something iconic: a selective retrieval by content 🔥. The last two sections wrap everything up with high-level functions and some conclusions 👑.
4.1 Objects ✨
An object has a name, a description, and a value. It can be related to other objects 🎀
The name is a symbol, which can be used to access the object from language functions 🔑
The description is a specification of properties of the object, many of which relate to the meaning or the representation of the value 📝
The value is the information of ultimate interest in the object 💖
The relationships between objects are hierarchical 🌳 Each object can be related directly to at most four other objects, designated as its parent, its child, its left_sibling, and its right_sibling 👯♀️
This specific concept of relationship is all that has been built into the model to date. One of our primary objectives in the future is to experiment with more general relationships among objects 🔮
4.2 Descriptions 📋
A description has the components name, type and type-dependent_parameters. It can be related hierarchically to other descriptions, according to a scheme similar to the one described for objects in 4.1 💅
The name has a role in referencing, just like it does for objects 🫶
Type is an undefined, intuitive idea for which we expect to develop a precise meaning within datalanguage(see section 3.10 for some of the ideas about this). In terms of the present model, it simply means one of the following: LIST, STRUCT, STRING, BOOL, DESC, DIR, FUNC, 0PD. Each of these refers to a sort of value corresponding to common ideas in programming (with the exception of OPD, which is explained in section 4.7), and on which certain operations are defined 🧠✨
Examples of type-dependent_parameters are the two items needed to define a STRING: size option and size. A STRING is a sequence of characters; the size of the STRING is the number of characters in it. If a STRING has a fixed size, then size option is FIXED and size is the number of characters it always contains. If a STRING has a varying size, then size option is VARYING, and size is its maximum (clearly, it might also have a minimum in a more refined scheme) 📏
When the description of an object has a type of STRING, it is commonly said that the object is a STRING 🧵
4.3 Values 💜
The value is the data itself 💎
An object of type BOOL can have only either the value TRUE or the value FALSE. Iconic binary energy 😭
An object of type STRING has values such as 'ABC', 'JOHN', or 'BOSTON' 🍓
Each value has a representation, in bits. Thus a BOOL is represented by a single bit, which will be a 'one' to represent TRUE and a 'zero' to represent FALSE 💻
4.4 Some examples 🎨
Here are some examples of structures involving objects, descriptions, and values. In these explanations and drawings, the objective is to convey some ideas about these primitive structures, so considerable detail is omitted in the drawings in the interest of clarity 🙈
Figure 4-1 shows two objects. X is of type string and has value 'ABC'. Y is of type bool and has value TRUE 🦋
_________________
| |
| _____________ |
| | X | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | STRING | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| VALUE | | ____________
|___________|_____| | |
OBJECT |____________\| "ABC" |
/|____________|
VALUE
_________________
| |
| _____________ |
| | Y | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | BOOL | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| VALUE | | ____________
|___________|_____| | |
OBJECT |____________\| TRUE |
/|____________|
VALUE
Figure 4-1
Two elementary objectsFigure 4-2 illustrates an object of type dir (a directory) and related objects. The directory has name SMITH. There are two objects entered in this directory, named X and Y 📂
_________________
| _____________ |
| | SMITH | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | DIR | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| CHILD | |
|___________|_____|
OBJECT |
___________V_____
| _____________ |
| | X | |
| |_____________| |
| NAME | _________________
| _____________ | | _____________ |
_____|_|____ | | | | Y | |
| | |_____________| | | |_____________| |
| | DESCRIPTION | | NAME |
| | _____________ | | _____________ |
| __|_|____ | | | | ____|_|_____
| | | |_____________| | | |_____________| | |
| | | VALUE | | DESCRIPTION | |
| | | _____________ | | _____________ | |
| | | | ____|_|_____\| | ____|_|__ |
| | | |_____________| | /| |_____________| | | |
| | | SIBLING | | VALUE | | |
| | |_________________| |_________________| | |
| | OBJECT OBJECT | |
| | _________________ _________________ | |
| |_\| "ABC" | | FALSE |/_| |
| /|_________________| |_________________|\ |
| VALUE VALUE |
| _________________ _________________ |
| | _____________ | | _____________ | |
| | | STRING | | | | BOOL | | |
|____\| |_____________| | | |_____________| |/____|
/| TYPE | | TYPE |\
|_________________| |_________________|
DESCRIPTION DESCRIPTION
Figure 4-2: A directory with two membersThe idea of a dir is similar to the idea of a file directory in most systems. A directory is a place where one can store named objects, freely adding and deleting them. The entries in the directory are all objects whose parent is that directory 🗂️✨
Figure 4-3 shows a more rigidly structured group of objects. Here we have R, a struct, and A and B, a pair of strings. Note that the boxes labeled 'object' in figure 4-3 bear precisely the same relationships to one another as those labeled 'object' in 4-2. However, there are two conditions which hold for 4-3 but do not hold for 4-2: (1) the value of R contains the values of A and B, and (2) the descriptions of R, A and B are all related 👑
Structs are the strict, rule-following sibling of directories 🧵 Keep an eye on the properties below!
Structs have the following properties: (1) name and description of each component in the struct is established when the struct is created, and (2) in a value of the struct, the order of occurrence of component
okay so the order of the component values is fixed, and that's literally the whole point 💅✨ here's the picture 🎀
_________________ _________________
| _____________ | | _____________ |
| | R | | | | STRUCT | |
| |_____________| | | |_____________| |
| NAME | | TYPE |
| _____________ | | _____________ |
| | ____|_|______\| | | |
| |_____________| | /| |__________|__| |
| DESCRIPTION | | CHILD | |
| _____________ | |____________|____|
_____|_|____ | | DESCRIPTION |
| | |_____________| | ____________V____
| | VALUE | | _____________ |
| | _____________ | | | STRING | |
| | | | | | |_____________| |
| | |_________|___| | ___\| TYPE | _____________
| | CHILD | | | /| _____________ | | _________ |
| |___________|_____| | | | ____|_|______\| | STRING | |
| OBJECT | | | |_____________| | /| |_________| |
| | | | SIBLING | | TYPE |
| ___________V_____ | |_________________| |_____________|
| | _____________ | | DESCRIPTION DESCRIPTION A
| | | A | | | |
| | |_____________| | | _________________ |
| | NAME | | | _____________ | |
| | _____________ | | | | B | | |
| | | ____|_|__| | |_____________| | |
| | |_____________| | | NAME | |
| | DESCRIPTION | | _____________ | |
| | _____________ | | | ____|_|___________________|
| __|_|____ | | | |_____________| |
| | | |_____________| | | DESCRIPTION |
| | | VALUE | | _____________ |
| | | _____________ | | | ____|_|____
| | | | ____|_|______\| |_____________| | |
| | | |_____________| | /| VALUE | |
| | | SIBLING | | _____________ | |
| | |_________________| | | | | |
| | OBJECT | |_____________| | |
| | | SIBLING | |
| | |_________________| |
| |__________ OBJECT _____________|
| _______|__________________________|_______
|____\| _____V_______ _______V_____ |
/| | "ABC" | | FALSE | | Figure 4-3
| |_____________| |_____________| | A STRUCT with
|__________________________________________| two membersokay now the big question, the Figure 4-4 list 🌸 it's called L and it's giving variable length vibes. here a similar structure of objects is implied, but because the structure is so regular, not every box labeled 'object' is actually drawn in there. lowkey smart 🧠
_________________
| _____________ |
| | L | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | LIST | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | | ________ |
| | | | | | | |
| |_______|_____| | | |______|_| |
| VALUE | | | CHILD | |
|_________|_______| |________|___|
OBJECT | DESCRIPTION |
| |
_________V_______ ________V___
| | | ________ |
| _____________ | | | STRING | |
| | "ABC" | | | |________| |
| |_____________| | | TYPE |
| _____________ | |____________|
| | "XY" | | DESCRIPTION
| |_____________| |
| _____________ |
| | "ZLM" | |
| |_____________| |
| : |
| : |
| _____________ |
| | "BBBF" | |
| |_____________| |
|_________________|
VALUE
Figure 4-4
A LISTso L has a variable number of components, and every single one of them has to satisfy the description subordinate to L's description. no cap, that's the rule 🦋
we could imagine a little 'object' box for every string in L 🧚 each box would point to its own string and to the shared description of those strings. but instead we just think in terms of creating those boxes whenever we actually need them. efficient and iconic 💖
4.5 Definitions of types ✨
ok so here are some more precise definitions of types in terms of the present model 🎀 these are here to lock in the semantics of our structure of objects, descriptions and values, but they should not be treated as a definition for the finished language specification, bestie 🙅♀️
An object of type STRING has a value which is a sequence of characters (figure 4-1) 🍓
An object of type BOOL has a value which is a truth value (TRUE or FALSE -- figure 4-1) 💜
An object of type DIR has subordinate objects, each with its own description and value. subordinate objects can be added and deleted at will (figure 4-2) 🌈
An object of type STRUCT has subordinate objects. each one has a description that is subordinate to the STRUCT's description, and a value contained in the STRUCT's value. the number, order and description of components is fixed when the STRUCT is created (figure 4-3) 👑
An object of type LIST may be thought of as having imaginary subordinate objects, and their existence is simulated by the techniques used when processing the LIST. each of these has the same description, which is subordinate to the description of the LIST. each has a distinct value, contained in the value of the LIST. in fact, only the LIST object, the LIST and component descriptions, and the values actually exist (figure 4-4) 🫶
An object of type DESC has a description as its value. this value is the same sort of entity that serves as the description of other objects 🦋
An object of type FUNC has a function call as its value. we'll be able to say more about this once functions have been discussed ⭐
An object of type OPD has an operation descriptor as its value (see 4.7 for details) 🔥
4.6 Object environment 🌸
there are three categories of objects in the model datacomputer 💅 they are p/objects, t/objects, and i/objects.
P/objects are permanent objects, created explicitly with language functions. they correspond to the idea of stored data in the real datacomputer. there are three special objects, and they're only special because they get created as part of initializing the environment, not as the result of running a language function. they're named STAR, BLOCK and TOP/LEVEL, and all three are of type DIR 👑
an object is a p/object if it is subordinate to STAR, and it is a t/object if it is subordinate to BLOCK. TOP/LEVEL is subordinate to BLOCK (see figures 4-5 and 4-6) 🧚
_________________
| |
| _____________ |
| | STAR | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | DIR | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| CHILD | |
|___________|_____|
OBJECT |
|
|
|
V
ALL P/OBJECTS
Figure 4-5
STAR and p/objectsT/objects are temporary objects, also created explicitly with language functions. the twist is that these correspond to user-defined temporaries, both local to requests and "top-level" (i.e. not local to any request, but existing until deletion or logout.) 🦋
_________________
| |
| _____________ |
| | BLOCK | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | DIR | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| VALUE | |
|___________|_____|
OBJECT |
|
|
___________V_____
| |
| _____________ |
| | TOP/LEVEL | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | DIR | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | ____|_|___ DESCRIPTION
| |_____________| | |
| SIBLING | |
| _____________ | |___\ ALL BLOCKS AND
| | | | / LOCAL T/OBJECTS
| |_________|___| |
| CHILD | |
|___________|_____|
|
|
V
ALL GLOBAL
T/OBJECTS
Figure 4-6
BLOCK, TOP/LEVEL and t/objectsI/objects are internal, system-defined objects. their creation and deletion happens implicitly when some language function runs ✨
I/objects hang directly off of function calls (objects of type FUNC), and they're always local to the execution of those function calls. they correspond to the notions of (1) literal, and (2) compiler- or interpreter-generated temporary 💖
4.7 Primitive Language Functions 🎀
here we talk about the primitive language functions that are currently implemented in the model and are probably the most interesting. in this section the emphasis is on relating the functions to one another. section 4.8 has more detail and examples 💅
Assign operates on a pair of objects, called the target and the source. the value of the source is copied into the value of the target. figure 4-7 shows a pair of objects, X and Y, before and after an assignment runs with X as target and Y as source 🍓 right now, assignment is only defined for objects of type BOOL and objects of type
STRING. ✨ Okay so the objects involved must have identical descriptions, no cap 💅
Figure 4-7 is the visual moment 🦋. It shows two objects, X and Y, before and after an assignment where X is the target and Y is the source. Before, X holds "ABC" and Y holds "DEF". After, both of them hold "DEF" 💖
_________________ _________________
| | | |
| _____________ | | _____________ |
| | | | | | | |
| | X | | | | Y | |
| |_____________| | | |_____________| |
| NAME | | NAME |
| _____________ | | _____________ |
| | | | | | | |
| |_______|_____| | | |_______|_____| |
| VALUE | | | VALUE | |
|_________|_______| |_________|_______|
OBJECT | OBJECT |
| |
_________V_______ _________V_______
| | | |
| "ABC" | | "DEF" |
|_________________| |_________________|
VALUE VALUE
BEFORE ASSIGNMENT
_________________ _________________
| | | |
| _____________ | | _____________ |
| | | | | | | |
| | X | | | | Y | |
| |_____________| | | |_____________| |
| NAME | | NAME |
| _____________ | | _____________ |
| | | | | | | |
| |_______|_____| | | |_______|_____| |
| VALUE | | | VALUE | |
|_________|_______| |_________|_______|
OBJECT | OBJECT |
| |
_________V_______ _________V_____
| | | |
| "DEF" | | "DEF" |
|_________________| |_________________|
VALUE VALUE
AFTER ASSIGNMENT
Figure 4-7
Effect of assignmentOkay, so listops are a whole class of primitive functions for manipulating LISTs 🧚 They're called _listops_, and every single listop takes in a special object called an _operation_descriptor_, or OPD 💜
To finish a complete operation on a LIST, you need a sequence of listops. There are semantic restrictions on how those sequences can be put together, so it's best to think of the whole sequence as one big operation. The state of that operation is kept in the OPD 🎀
Look back at figure 4-4 🌸 There's one box labeled "object", and it represents the list as a whole. To operate on any given member, we need an object box for that member too, and figure 4-8 adds exactly that. Its value is one of the components of the LIST value, and its description is subordinate to the LIST description. In 4-8, this object is named M ✨
_________________ _________________
| | | _____________ |
| _____________ | | | | |
| | L | | | |_____________| |
| |_____________| | | TYPE |
| NAME | | _____________ |
| _____________ | | | | |
| | ____|_|______\| |__________|__| |
| |_____________| | /| CHILD | |
| DESCRIPTION | |____________|____|
| _____________ | DESCRIPTION |
| | | | |
| |_________|___| | ____________V____
| VALUE | | | _____________ |
|___________|_____| | | STRING | |/___
OBJECT | | |_____________| |\ |
| | TYPE | |
___________V_____ |_________________| |
| | DESCRIPTION |
| _____________ | |
| | "ABC" | | _________________ |
| |_____________| | | | |
| _____________ | | _____________ | |
| | "XY" | | | | M | | |
| |_____________| | | |_____________| | |
| _____________ | | NAME | |
| | "ZLM" |/|___ | _____________ | |
| |_____________|\| | | | ____|_|____|
| : | | | |_____________| |
| : | | | DESCRIPTION |
| _____________ | | | _____________ |
| | "BBBF" | | |___|_|____ | |
| |_____________| | | |_____________| |
|_________________| | VALUE |
VALUE |_________________|
OBJECT
Figure 4-8
LIST and member objectsIn 4-8 we have enough structure to give M a description and a value, and that's enough to run operations on M as an item 🙌 BUT there's no direct link between object M and object L 🙈 That's where figure 4-9 comes in, because the structure gets completed by adding an OPD.
_________________ _________________
| _____________ | | _____________ |
| | L | | | | | |
| |_____________| | | |_____________| |
| NAME | | TYPE |
| _____________ | | _____________ |
| | ____|_|______\| | | |
| |_____________| | /| |__________|__| |
| DESCRIPTION | | CHILD | |
| _____________ | |____________|____|
| | | |/__ DESCRIPTION |
| |_________|___| |\ | ____________V____
| VALUE | | | | _____________ |
|___________|_____| | | | STRING | |/___
OBJECT | | | |_____________| |\ |
| | | TYPE | |
___________V_____ | |_________________| |
| _____________ | | DESCRIPTION |
| | "ABC" | | | _________________ |
| |_____________| | | | | |
| _____________ | | | _____________ | |
| | "XY" | | |___|_|____ | | |
| |_____________| | | |_____________| | |
| _____________ | | LIST | |
| | "ZLM" | | | _____________ | |
| |_____________| | | | | | |
| : | | |_________|___| | |
| : | | MEMBER | | |
| _____________ | | : | | |
| | "BBBF" |/|___ | : | | |
| |_____________|\| | |___________|_____| |
|_________________| | OPD | |
VALUE | ___________V_____ |
| | _____________ | |
| | | M | | |
| | |_____________| | |
| | NAME | |
| | _____________ | |
| | | ____|_|____|
| | |_____________| |
| | DESCRIPTION |
| | _____________ |
|___|_|____ | |
| |_____________| |
Figure 4-9 | VALUE |
OPD, LIST and member |_________________|
OBJECTThe OPD sets up the object relationship, and it also holds info about the sequence of primitive listops in progress 🔥 Once enough info lives in the OPD, 4-9 becomes a structure that's solid for keeping the LIST and the whole list operation intact 👑 Besides the LIST and member pointers, the OPD records:
- (1) which suboperations are enabled for the sequence
- (2) the current suboperation
- (3) the instance number of the current LIST member
- (4) an end-of-list indicator
The suboperations are add/member, delete/member, change/member and get/member, and they all apply to the current member 💅 Only suboperations that were enabled at the beginning of a sequence can be executed during that sequence. Eventually, that advance knowledge of intentions will give important info for concurrency control and optimization 🧠✨
Right now, an OPD relates one member object to one LIST object. That's a real restriction on the kinds of operation sequences you can express 💀 Any LIST transformation that needs simultaneous access to more than one member has to be represented as more than one sequence. (And we don't yet solve the problems that come with running such sequences concurrently, even when one process controls both 🙃)
You can still transform a LIST by parking intermediate results in temporary objects. Honestly though, it's way more desirable to build multiple current members into the semantics of the language than to lean on temporaries 🙄 An important future extension of the listops will deal with this problem 🦋
There are six listops:
listop/beginlistop/endwhich/memberend/of/listopen/memberclose/member
listop/begin and listop/end do the obvious jobs of starting and terminating a sequence of listops 🎀 listop/begin takes in LIST and member objects, an OPD, and a specification of the suboperations to enable. It initializes the OPD, including setting up the links to the LIST and MEMBER objects. After the OPD-LIST-member relationship is established, you only need to pass the OPD and auxiliary parameters to a listop in the sequence, because everything else can be derived from the OPD 🧚
listop/end clears the OPD and frees any resources acquired by listop/begin 🧹
which/member establishes the current member for any suboperations. That's either the first LIST member, the last LIST member, or the next LIST member. This listop only identifies which member is going to be operated on. It does not make the contents of the member accessible 👀
open/member and close/member bracket a suboperation, and the suboperation is given as an argument to open/member. open/member always establishes a pointer from the member object to the member value, and close/member always clears that pointer. Each of these listops may also take some action, depending on the suboperation 💖
The details of that action depend on how the LIST is represented in storage, how big a LIST member is, and the choices made in implementation 🤷♀️
Between open/member and close/member, the data is accessible. It can always be read, and for the add/member and change/member suboperations, it can also be written into 📖✍️
end/of/list tests a flag in the OPD and returns an object of type BOOL. The value of that object is the same as the value of the flag. It is TRUE if a get/member, change/member or delete/member would be unsuccessful because a which/member has moved "beyond the end" 🏁 This listop is provided so that you can write procedures that terminate conditionally once all members have been processed 🫶
4.8 Details of primitive language functions ✨
Okay so get/struct/member is the one that handles STRUCTs 💖 You give it a STRUCT object that points to the STRUCT value, and it sets up a pointer from a given member object over to the member value 🦋 That pointer is the dashed line in figure 4-10, so look for it 👀
_________________ _________________
| _____________ | | _____________ |
| | F | | | | STRUCT | |
| |_____________| | | |_____________| |
| NAME | | TYPE |
| _____________ | | _____________ |
| | ____|_|______\| | | |
| |_____________| | /| |__________|__| |
| DESCRIPTION | | CHILD | |
| _____________ | |____________|____|
| | | | DESCRIPTION |
| |___________|_| | ____________V____ _________________
| VALUE | | | _____________ | | _____________ |
| ___________|_ | | | STRING | | | | STRING | |
| | | | | | |_____________| | | |_____________| |
| |_________|_|_| | | TYPE | | TYPE |
| CHILD | | | | _____________ | | _____________ |
|___________|_|___| ____\| | | | | | | |
OBJECT | | | /| |_____________| | | |_____________| |
| | | | SIBLING | | SIBLING |
| | | |_________________| |_________________|
| | | DESCRIPTION DESCRIPTION A
| | | ______________________________________ |
| | | | ____________ ____________ | |
| | | | | "ABC" | | FALSE | | |
| |_____|_____| |____________| |____________| | |
| | |________A_____________________________| |
| | ............: VALUE |
___________V_____ | : _________________ |
| _____________ | | : | _____________ | |
| | A | | | : | | B | | |
| |_____________| | | : | |_____________| | |
| NAME | | : | NAME | |
| _____________ | | : | _____________ | |
| | ____|_|_| : | | ____|_|_______________________|
| |_____________| | : | |_____________| |
| DESCRIPTION | : | DESCRIPTION |
| _____________ | : | _____________ |
| | ....|.|....: | | | |
| |_____________| | | |_____________| |
| VALUE | | VALUE |
| _____________ | | _____________ |
| | ____|_|______\| | | |
| |_____________| | /| |_____________| |
| SIBLING | | SIBLING |
|_________________| |_________________| Figure 4-10
OBJECT OBJECT Effect of GET/STRUCT/MEMBERThe primitives we've covered so far (assign, listops, and get/struct/member) give us a basic toolkit for working with structures made of LISTs, STRUCTs and elementary items 🧚 With just those, you can move the contents of one hierarchical structure into another, stick structures together, delete chunks of structures, and so on ✨ But for the really fun stuff, we need facilities for control and selection 💅
We get a lil bit of control from if/then, if/then/else, till and while 🌸 Each one evaluates one primitive function call, and that call has to return a BOOL. Whatever that BOOL says decides what happens next 🫶
Say A and B are function calls 🎀 if/then(A,B) runs B if A returns TRUE. if/then/else(A,B,C) runs B if A returns TRUE, and runs C if A returns FALSE. while and till both iterate, running A first and then B. while ends the loop when A returns FALSE, and till ends it when A returns TRUE. If that happens the very first time, B never runs at all 😭
So far we've only met one function that returns a BOOL: the listop end/of/list 👀 Two other classes do it too: the booleans and the comparisons 💜 There are 3 primitive booleans (and, or, not) and six primitive comparisons (equal, less/than, greater/than, not/equal, less/than/or/equal, greater/than/or/equal). Only equal is implemented at time of publication 🙈
The booleans take BOOLs in and give BOOLs out. The comparisons take a pair of elementary objects with the same description and give a BOOL out 🔥 Expressions made from booleans and comparisons on item contents are one of the principal tools for selectively referencing data in data management systems 😤
With the booleans, the comparisons, and the primitives from before, we can do selective "retrievals" ✨ For example, we can transfer to LIST B all items in LIST A having a value of 'ABC' 🧚 In fact, we now have a (semantically) general ability to do content-based retrievals and updates on arbitrary hierarchical structures 👑 We can even program something as complex as processing a list of transactions against a master list, which is one of the typical applications in business data processing 💼
Of course, we would not expect users of datalanguage to write requests at the listop level 💀 The listops defined here also aren't a very efficient way to do some of these tasks 🙈 To get good solutions, we need higher-level operators and other primitives that use other techniques in processing 🌈
Besides the ones we've already discussed, the model has functions for: (1) referencing an object by qualified name, (2) generating a constant, (3) generating data descriptions, (4) writing compound functions and blocks with local variables, and (5) creating objects 🍓
The facilities for generating constants and data descriptions (and data descriptions are a special case of constants) are kinda marginal, with no features of special interest 🙈 Data description will be a big deal later on in the modeling effort, though 🫶
Object referencing functions let you reference t/objects and p/objects (these terms are defined in 4.6, bestie 📖). A p/object is referenced by giving the pathname from STAR to it. A t/object is referenced by giving the pathname from the block directory in which it is defined 🦋
compound/function lets a sequence of function calls be treated syntactically as a single call 💅 So, for example, in if/then(A,B), B is frequently a call to compound/function, which in turn calls a sequence of other functions 🎀
create takes two inputs: a superior object and a description 👑 The superior must be a directory. The new object is created as the leftmost child of the directory, and its name is determined by the description ✨
Here we get the specs for the primitives from the last section 📚 We're still leaving out details when we judge them to be of no general interest, because the goal is to give you enough info to examine examples yourself 🧚
Most of the primitives occur at two levels in the model 🌸 The internal primitives are called i/functions, and the external, or language primitives, are called l/functions. The relationship between the two is explained in 4.9. In this section we're talking about i/functions 💜
L/functions take in and give out forms, which are tree structures whose terminal nodes are atoms 🌳 Atoms are things like function names, object names, literal string constants, truth values and delimiters. Calls to i/functions are also expressed as forms 🎀
Any form can be evaluated, yielding some object ✨ A form that is an i/function call yields the value that the i/function returns, which is another form 🔁 In general, the form returned by an i/function call will, when evaluated, yield a datalanguage object (the sort of object we've been representing with an "object box" in the drawings 📦).
4.8.1 Name recognition functions 🎀
These return a form which evaluates to an object 💖
L/TOBJ
Input must name a temporary object subordinate either to TOP/LEVEL or a block directory 🧚
L/POBJ
Input must name a permanent object (aka an object subordinate to STAR) 💅
Typical calls are L/POBJ(X.Y.Z) and L/TOBJ(A) 🌈
4.8.2 Constant generators 🧁
Each of these inputs an atomic symbol and yields a value for a constant to be created 💜 Each one returns a form that will evaluate to an object having the specified value and an appropriate description 🫶
LC/STRING- a typical call isLC/STRING('ABC')LC/BOOL- a typical call isLC/BOOL(TRUE)
4.8.3 Elementary item functions 🧩
These input and output forms that evaluate to elementary objects (objects which can have no subordinate object, basically objects whose value is treated as atomic) 🔍 Eventually all the comparison operators will be implemented, babe 🔜
L/ASSIGN
Inputs must evaluate to either STRINGs or BOOLs 🎀 It outputs a form that transfers the value of the second input to the first 💌 Typical call:
L/ASSIGN(L/TOBJ(A),LC/STRING('XYZ'))When the output form is evaluated, it copies 'XYZ' into A's value ✨
L/EQUAL
Inputs are a pair of forms evaluating to objects, which must have identical descriptions and be BOOLs or STRINGs 🔍 It returns a form evaluating to an object of type BOOL. The value of this object is TRUE if the inputs have identical descriptions and values, and otherwise it is false 🙅 Typical call:
L/EQUAL(L/TOBJ(X),LC/STRING('DEF'))L/AND, L/OR, L/NOT
These are the standard boolean operators 💜 Inputs are forms evaluating to BOOLs, and the output is a form evaluating to a BOOL 🫶 L/AND and L/OR take two inputs, while L/NOT takes one 💅 Typical call:
L/AND( L/EQUAL(L/TOBJ(X),LC/STRING('DEF')),
L/EQUAL(T/TOBJ(Y),LC/STRING('GHI')) )The form returned will, when evaluated, return TRUE if both X has the value 'DEF' and Y has the value 'GHI' 🎉
4.8.4 Data description functions 📐
These all return a form evaluating to a description (i.e. the thing that our drawings represent with a box labeled "description") 💕
LD/STRING
Inputs 3 parameters specifying the name, size option and size for the string 🧵 Typical call:
LD/STRING(X,FIXED,3)This call returns a form evaluating to a description for a fixed-length 3-character string named X 🎀
LD/LIST
Inputs two forms. The first is the name of the LIST, and the second evaluates to a description of the LIST member 💕 Typical call:
LD/LIST(L,LD/STRING(M,FIXED,3))It creates the structure shown in figure 4-11, and returns a form evaluating to the description represented by the upper box 🏠
_________________
| _____________ |
| | L | |
| |_____________| |
| NAME |
| _____________ |
| | LIST | |
| |_____________| |
| TYPE |
| _____________ |
| | | |
| |________|____| |
| CHILD | |
|__________|______|
DESCRIPTION |
|
__________V______
| _____________ |
| | M | |
| |_____________| |
| NAME |
| _____________ |
| | STRING | |
| |_____________| |
| TYPE |
| _____________ |
| | _________ | |
| | | FIXED | | |
| | |_________| | |
| | _________ | |
| | | 3 | | |
| | |_________| | |
| |_____________| |
| PARAMETERS |
|_________________|
DESCRIPTION
Figure 4-11
LIST and member descriptionsLD/STRUCT
The STRUCT one is giving main character energy 💅. You hand it a name for the STRUCT plus one or more forms that each evaluate to a description, and those descriptions become the members 🎀 Typical call:
LD/STRUCT(R,
LD/STRING(A,FIXED,3)
LD/BOOL(B) )It builds the structure in 4-12 and gives you back a form that evaluates to the top box ✨
_________________
| _____________ |
| | R | |
| |_____________| |
| NAME |
| _____________ |
| | STRUCT | |
| |_____________| |
| TYPE |
| _____________ |
| | | |
| |_________|___| |
| CHILD | |
|___________|_____|
DESCRIPTION |
|
___________V_____
| _____________ |
| | A | |
| |_____________| |
| NAME |
| _____________ |
| | STRING | |
| |_____________| |
| TYPE | _________________
| _____________ | | _____________ |
| | | | | | B | |
| |_____________| | | |_____________| |
| PARAMETER | | NAME |
| _____________ | | _____________ |
| | ____|_|______\| | BOOL | |
| |_____________| | /| |_____________| |
| SIBLING | | TYPE |
|_________________| |_________________|
DESCRIPTION DESCRIPTION
Figure 4-12
STRUCT and member descriptionsLD/BOOL, LB/DIR, LD/OPD, LD/FUNC, LD/DESC 🌈
These all do the same kind of thing: each one takes a name and makes a single description, then returns a form that evaluates to the description it made. Typical call:
LD/BOOL(X)4.8.5 Data creation 🦋
L/CREATE 💖
This one takes two forms and evaluates them. The first must yield an object of type DIR, and the second must yield a description for the object being created. It creates the object and returns a form which, when evaluated, will generate a value for the new object. A simple example:
L/CREATE(L/TOBJ(X),LD/B0OL(Y))Figure 4-13 shows the directory X before that call runs. It only contains an OPD. After execution, the directory looks like 4-14. Creating a value for Y happens when the form returned by L/CREATE gets evaluated (covered in section 4.9) 🫶
_________________
| |
| _____________ |
| | X | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | DIR | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| CHILD | |
|___________|_____|
OBJECT |
|
|
___________V_____
| |
| _____________ |
| | Z | |
| |_____________| |
| NAME | ____________
| _____________ | | ________ |
| | ____|_|______\| | OPD | |
| |_____________| | /| |________| |
| DESCRIPTION | | TYPE |
| _____________ | |____________|
| | | | DESCRIPTION
| |_________|___| |
| VALUE | | ____________
|___________|_____| | |
OBJECT |____________\| |
/|____________|
OPD
Figure 4-13
X and Z before creation of Y _________________
| _____________ |
| | X | |
| |_____________| | _________________
| NAME | | _____________ |
| _____________ | | | DIR | |
| | ____|_|______\| |_____________| |
| |_____________| | /| TYPE |
| DESCRIPTION | |_________________|
| _____________ | DESCRIPTION
| | | |
| |_________|___| |
| VALUE | |
|___________|_____|
OBJECT |
___________V_____
| _____________ |
| | Y | |
| |_____________| | _________________
| NAME | | _____________ |
| _____________ | | | BOOL | |
| | ____|_|______\| |_____________| |
| |_____________| | /| TYPE |
| DESCRIPTION | |_________________|
| _____________ | DESCRIPTION
| | | |
| |_____________| |
| VALUE |
| _____________ |
| | ____|_|______________
| |_____________| | |
| SIBLING | |
|_________________| ______V__________ _________________
OBJECT | _____________ | | _____________ |
| | Z | | | | OPD | |
| |_____________| | __\| |_____________| |
| NAME | | /| TYPE |
| _____________ | | |_________________|
| | ____|_|___| DESCRIPTION
| |_____________| |
| DESCRIPTION |
| _____________ | _________________
| | ____|_|______\| |
Figure 4-14 | |_____________| | /|_________________|
X, Y, and Z after | VALUE | OPD
L/CREATE |_________________|
OBJECT4.8.6 Control 🎀
L/IF/THEN, L/IF/THEN/ELSE 🔥
Used to request conditional evaluation of a form. Typical call:
L/IF/THEN(L/EQUAL(L/TOBJ(A),LC/STRING('ABC'),
L/ASSIGN(L/TOBJ(B),LC/STRING('DE')))The form returned does the following when evaluated: if A has value 'ABC', then it stores 'DE' in the value of B 👑
L/WHILE, L/TILL 🌸
These iterate conditionally, as explained in the previous section. Examples appear later 🧚
L/CF 💅
Compound function: it takes one or more forms and returns a form which, when evaluated, evaluates each input in sequence. Typical call:
L/CF(L/ASSIGN(L/TOBJ(R.A),LC/STRING('XX')),
L/ASSIGN(L/TOBJ(R.B),LC/STRING('YY')))When the output of L/CF is evaluated, it assigns new values to R.A and R.B 🫶
4.8.7 Listops 🌈
These primitives get run in sequences, in order, to do operations on LISTs. Except for L/END/OF/LIST, these functions output forms that are evaluated for effect only, which means the output forms don't return values themselves 💖
L/LISTOP/BEGIN ✨
Takes forms evaluating to (1) a LIST, (2) an object to represent the current LIST member, and (3) an OPD. It also takes a list of atomic forms whose values are taken as suboperations to enable. Typical call:
L/LISTOP/BEGIN(L/POBJ(F),L/TOBJ(R),
L/TOBJ(OPF),ADD,DELETE)This returns a form that initializes a sequence of listops to run on F. The caller has already created R and OPF, and intends to ADD and DELETE list members 🦋
All subsequent calls in this sequence of listops only need to specify the OPD and auxiliary parameters 🎀
L/LISTOP/END 🔥
Takes a form evaluating to an OPD. Outputs a form which, when evaluated, clears the OPD and breaks the relationships between the OPD, the LIST, and member objects 🙈
L/WHICH/MEMBER 💜
Takes two forms. The first evaluates to an OPD; the second is one of FIRST, LAST, NEXT. The output form, when evaluated, establishes a new current member for the next suboperation. Note: this does not make the value of the member accessible, it just identifies it by setting the instance number in the OPD 👀 A typical call:
L/WHICH/MEMBER(L/TOBJ(OPF),NEXT)When a which/member causes an advance beyond the end of the list, a flag is set in the OPD 🚩
L/END/OF/LIST 🍓
Takes a form evaluating to an OPD. Outputs a form which, when evaluated, returns a BOOL. It has value TRUE if the end of list flag in the OPD is on 🌈
L/OPEN/MEMBER 🧚
Takes a form evaluating to an OPD and a form which must be one of ADD, DELETE, GET, CHANGE. Outputs a form which, when evaluated, starts the requested suboperation on the current LIST member. The suboperation always establishes the pointer from the member object to the current member value instance. In addition, in the case of ADD, this value must be created. Typical call:
L/OPEN/MEMBER (L/TOBJ (OPF) ,ADD)L/CLOSE/MEMBER 💅
Takes a form evaluating to an OPD. Outputs a form which, when evaluated, completes the suboperation in progress. A typical call:
L/CLOSE/MEMBER(L/TOBJ(OPF))It always clears the pointer from the member object to the member value. In addition, in the case of DELETE, it removes the member value from the LIST. In the case of ADD, it enters the member value in the LIST. It also makes the member added the current member, so a sequence of ADDs run without intervening which/members will add the new members in sequence 🫶
An elaborate example, involving listops and several other primitives, appears in section 4.10 🌟
4.9 Execution cycle ⭐
The model datacomputer has a two-part execution cycle: it first compiles requests, then interprets them. A "request" is an l/function call. "Compilation" is the aggregate result of executing all the l/function calls involved in the request. That's typically many calls, since there are usually several levels of nested calls, with the results of the inner calls being delivered as arguments to the next level of calls 🌸 Usually, executing an l/function involves a simple macro expansion, preceded by some binding, checking and (eventually) optimization 💖
The compiled form consists wholly of atomic symbols and i/function calls. The i/functions are internal primitives which input and output datalanguage objects (the entities represented by the boxes labeled "object" in the drawings) 🧚
Each of the l/functions discussed compiles into a single i/function, so the macro expansion aspect of compilation is presently trivial. However, this will not be true in general; it's only that these are _primitive_ l/functions that make it true now 🔥
The decision to use a compile-and-interpret cycle needs some explanation. The way to understand it is to think in terms of the functions that would be performed in a strictly interpretive system. There would still be a requirement to perform global checks on the validity of the request in advance of executing any part of it. This is because partial execution of an incorrect request can leave a database in an inconsistent state, and if the database is large or complex, the cost of recovery will be considerable. So it pays to do as much checking as possible. When the system is fully developed, this will include a certain element of simple prediction of execution flow. In any case, much more than syntactic checking is implied 💀
Since any such global checks are performed in advance of actual execution, they are effectively not part of the execution itself, for any given form. By performing them as part of a separate compilation process, we only formalize a modularity which already effectively exists 🌈
There will still be cases, however, in which checking, binding and
optimization functions must be executed during interpretation, if at all 💅 bestie, this happens when the info they need isn't available until some of the data has already been accessed 👀. When it's practical, we handle these moments by designing most functions so they can run in either "half" of the cycle 🦋
As the model grows, we'll get a better feel for this whole problem ✨ It's honestly reasonable for us to end up with lots of cycles of compilation and interpretation, maybe even cycles nested inside other cycles 🌀 main character structure, no cap 👑
4.10. Examples of operations on LISTs ✨
okay so here we're gonna walk through an operation on a LIST using the primitive l/functions 🧚 We start by showing the function calls needed to create a LIST named F and give it a few member values. Then we selectively copy certain members over to a second LIST G 💖
To create F:
L/CREATE("STAR",LD/LIST(F,
LD/STRUCT(R,
LD/STRING(A,FIXED,2),
LD/STRING(B,FIXED,2))))This creates F as a member of the permanent directory STAR (see section 4.6 for details about STAR). The symbol STAR is lowkey special in the "language" 🎀 It's one of the few atomic symbols that evaluate directly to an object. (Recall that most permanent objects get referenced through a call to L/POBJ. Reserving the symbol STAR is equivalent to reserving STAR as a name and writing L/POBJ(STAR). The solution we picked here is just easier to write 🙈) Running this call builds the structure shown in 4-15 (except for STAR, which already existed before the call). The value that's initially created for F is an empty LIST, a LIST with zero members 🍓
_________________ _________________
| _____________ | | _____________ |
| | STAR | | | | F | |
| |_____________| | | |_____________| |
| NAME | | NAME |
| _____________ | | _____________ |
| | | | | | LIST | |
| |_________|___| | | |_____________| |
| CHILD | | | TYPE |
|___________|_____| | _____________ |
OBJECT | | | | |
| | |___________|_| |
___________V_____ __\| CHILD | |
| _____________ | | /|_____________|___|
| | F | | | DESCRIPTION |
| |_____________| | | |
| NAME | | _____________V___
| _____________ | | | _____________ |
| | ____|_| | | | R | |
| |_____________| |___| | |_____________| |
| DESCRIPTION | | NAME |
| _____________ | | _____________ |
| | | | | | STRUCT | |
| |_________|___| | | |_____________| |
| VALUE | | | TYPE |
|___________|_____| | _____________ |
OBJECT | | | | |
| | |___________|_| |
___________V_____ | CHILD | |
| | |_____________|___|
| | DESCRIPTION |
|_________________| _____________V___
VALUE | _____________ |
| | A | |
| |_____________| |
| NAME | _________________
| _____________ | | _____________ |
| | STRING | | | | B | |
| |_____________| | | |_____________| |
| TYPE | | NAME |
| _____________ | | _____________ |
| | ____|_|_____\| | STRING | |
Figure 4-15 | |_____________| | /| |_____________| |
F immediately after | SIBLING | | TYPE |
creation |_________________| |_________________|
DESCRIPTION DESCRIPTIONTo add members to F, we need listops, and for that we have to create two more objects: an object to represent the current member, and an operation descriptor (OPD) 🔥 These are temporaries rather than permanent objects, and they're also "top level" (meaning they're not local to a request). Temporary, top level objects get created as members of the directory TOP/LEVEL. The calls to create them are:
L/CREATE(L/TOBJ(TOP/LEVEL),
LD/STRUCT(M,
LD/STRING(A,FIXED,2),
LD/STRING(B,FIXED,2)))
L/CREATE(L/TOBJ(TOP/LEVEL),LD/OPD(OPF))We create M to represent the current member, and its description is the same as the one we input for a member of F (see the call that created F) 💅 The proper way to do this is with a mechanism that shares the actual LIST member description with M, but that mechanism doesn't exist in our model yet 😭
Now we wanna add some data to F. Each member is gonna be a STRUCT containing two two-character STRINGs 🌸
To begin the listop sequence:
L/LISTOP/BEGIN(L/POBJ(F),L/TOBJ(M),
L/TOBJ(OPF),ADD)This call sets up the structure shown in figure 4-16. It initializes the OPD, making it point to F and M, and it records that only the ADD suboperation is enabled ✨
_________________ _________________
| _____________ | | _____________ |
| | STAR | | | | OPF | |
| |_____________| | | |_____________| |
| NAME | | NAME |
| _____________ | | _____________ |
| | | | | | | |
| |_________|___| | | |_______|_____| |
| CHILD | | | VALUE | |
|___________|_____| |_________|_______|
OBJECT | OBJECT |
___________V_____ _________V______
| _____________ | | _____________ |
| | F | |/______|_|____ | |
| |_____________| |\ | |_____________| |
| NAME | | LIST |
| _____________ | | _____________ |
| | | | | | | |
| |_________|___| | | |________|____| |
| VALUE | | | MEMBER | |
|___________|_____| |__________|______|
OBJECT | VALUE | OPD
| __________V______
___________V_____ | _____________ |
| | | | M | |
| LIS | | |_____________| |
|_________________| | NAME |
VALUE | _____________ |
| | | |
| |________|____| |
| CHILD | |
|__________|______|
OBJECT |
__________V______ _________________
| _____________ | | _____________ |
| | A | | | | B | |
| |_____________| | | |_____________| |
| NAME | | NAME |
| _____________ | | _____________ |
| | ____|_|_____\| | | |
| |_____________| | /| |_____________| |
| SIBLING | | |
|_________________| |_________________|
OBJECT OBJECT
Figure 4-16
F, OPF and M after L/BEGIN/LISTOPNext up, we have to set a current member 🎀 We want to add members to the end (in this case, adding them anywhere would get the same effect, since the LIST is empty), and we do that by making LAST the current member:
L/WHICH/MEMBER(L/TOBJ(OP1),LAST)Now, to add a new member to F, we can run the following:
L/OPEN/MEMBER(L/TOBJ(OPF),ADD)
L/ASSIGN(L/TOBJ(M.A),LC/STRING('AB'))
L/ASSIGN(L/TOBJ(M.B),LC/STRING('CD'))
L/CLOSE/MEMBER(L/TOBJ(OPF))L/OPEN/MEMBER creates a STRUCT value for M. It does not affect the value of F. Each member of the STRUCT value gets initialized when the STRUCT is created. The result is shown in 4-17, and notice that the STRUCT member values aren't related to the objects M.A and M.B yet 🙈
Figure 4-18 shows the changes made by the first L/ASSIGN. The pointer from the object M.A to the value was set up by a GET/STRUCT/MEMBER that got compiled by L/TOBJ(M.A), and the value got filled in by the assign operator. The second assign does something similar and fills in the second value. The call to L/CLOSE/MEMBER takes the value shown for M in 4-18 (with the second member value filled in) and adds it to the value of F. The result is shown in 4-19, so compare it with 4-16 🦋
_________________ _________________
| _____________ | | _____________ |
| | M | | | | STRUC | |
| |_____________| | | |_____________| |
| NAME | | TYPE |
| _____________ | | _____________ |
| | ____|_|______\| | | |
| |_____________| | /| |__________|__| |
| DESCRIPTION | | CHILD | |
| _____________ | |____________|____|
_____|_|____ | | DESCRIPTION |
| | |_____________| | |
| | VALUE | ____________V____
| | _____________ | | _____________ |
| | | | | | | STRING | |
| | |_________|___| | | |_____________| |
| | CHILD | | ___\| TYPE | _____________
| |___________|_____| | /| _____________ | | _________ |
| OBJECT | | | | ____|_|______\| | STRING | |
| | | | |_____________| | /| |_________| |
| ___________V_____ | | SIBLING | | TYPE |
| | _____________ | | |_________________| |_____________|
| | | A | | | DESCRIPTION DESCRIPTION A
| | |_____________| | | |
| | NAME | | _________________ |
| | _____________ | | | _____________ | |
| | | ____|_|__| | | B | | |
| | |_____________| | | |_____________| | |
| | DESCRIPTION | | NAME | |
| | _____________ | | _____________ | |
| | | | | | | ____|_|___________________|
| | |_____________| | | |_____________| |
| | VALUE | | DESCRIPTION |
| | _____________ | | _____________ |
| | | ____|_|______\| | | |
| | |_____________| | /| |_____________| |
| | SIBLING | | VALUE |
| |_________________| |_________________|
| OBJECT OBJECTOkay bestie, here's the first diagram, Figure 4-17, showing the state right after L/OPEN/MEMBER 🎀
|___________________________
|
_____________________V____________________
| _____________ _____________ |
| | | | | |
| |_____________| |_____________| |
|__________________________________________| Figure 4-17
VALUE After L/OPEN/MEMBER
_________________ _________________
| _____________ | | _____________ |
| | M | | | | STRUC | |
| |_____________| | | |_____________| |
| NAME | | TYPE |
| _____________ | | _____________ |
| | ____|_|______\| | | |
| |_____________| | /| |__________|__| |
| DESCRIPTION | | CHILD | |
| _____________ | |____________|____|
_____|_|____ | | DESCRIPTION |
| | |_____________| | |
| | VALUE | ____________V____
| | _____________ | | _____________ |
| | | | | | | STRING | |
| | |_________|___| | | |_____________| |
| | CHILD | | ___\| TYPE | _____________
| |___________|_____| | /| _____________ | | _________ |
| OBJECT | | | | ____|_|______\| | STRING | |
| | | | |_____________| | /| |_________| |
| ___________V_____ | | SIBLING | | TYPE |
| | _____________ | | |_________________| |_____________|
| | | A | | | DESCRIPTION DESCRIPTION A
| | |_____________| | | |
| | NAME | | _________________ |
| | _____________ | | | _____________ | |
| | | ____|_|__| | | B | | |
| | |_____________| | | |_____________| | |
| | DESCRIPTION | | NAME | |
| | _____________ | | _____________ | |
| __|_|____ | | | | ____|_|___________________|
| | | |_____________| | | |_____________| |
| | | VALUE | | DESCRIPTION |
| | | _____________ | | _____________ |
| | | | ____|_|______\| | | |
| | | |_____________| | /| |_____________| |
| | | SIBLING | | VALUE |
| | |_________________| |_________________|
| | OBJECT OBJECT
| |___________
| |
| ________|_________________________________
| | ______V______ _____________ |
|____\| | "AB" | | | |
/| |_____________| |_____________| |
|__________________________________________| Figure 4-18
VALUE After first L/ASSIGN
_________________ _________________
| _____________ | | _____________ |
| | STAR | | | | OPF | |
| |_____________| | | |_____________| |
| NAME | | NAME |
| _____________ | | _____________ |
| | | | | | | |
| |_________|___| | | |___________|_| |
| CHILD | | | VALUE | |
|___________|_____| |_____________|___|
OBJECT | OBJECT |
___________V_____ _____________V___
| _____________ | | _____________ |
| | F | |/______|_|____ | |
| |_____________| |\ | |_____________| |
| NAME | | LIST |
| _____________ | | _____________ |
| | | | | | | |
| |_________|___| | | |___________|_| |
| VALUE | | | MEMBER | |
|___________|_____| |_____________|___|
OBJECT | VALUE | OPD
| _____________V___
______________V_________ | _____________ |
| ______________________ | | | M | |
|| _________ _________ || | |_____________| |
||| "AB" || "CD" ||| | NAME |
|||_________||_________||| | _____________ |
||______________________|| | | | |
| / | | |___________|_| |
| / | |_____________|___|
|_______________/________| OBJECT |
VALUE / / _____________V___ _________________
/ / | _____________ | | _____________ |
/ / | | | | | | B | |
/ LIST | |_____________| | | |_____________| |
/ | NAME | | NAME |
/ | _____________ | | _____________ |
NEW MEMBER VALUE | | ____|_|_____\| | | |
| |_____________| | /| |_____________| |
|_________________| |_________________|
OBJECT OBJECT
Figure 4-19
After L/CLOSE/MEMBERBy running similar groups of four primitives, changing only the values of the constants, we can build up the LIST F shown in 4-20. ✨ The calls you need look like this:
L/OPEN/MEMBER(L/TOBJ(OPF),ADD)
L/ASSIGN(L/TOBJ(M.A),LC/STRING('FF'))
L/ASSIGN(L/TOBJ(M.B),LC/STRING('GH'))
L/CLOSE/MEMBER(L/TOBJ(OPF))
L/OPEN/MEMBER(L/TOBJ(OPF),ADD)
L/ASSIGN(L/TOBJ(M.A),LC/STRING('AB'))
L/ASSIGN(L/TOBJ(M.B),LC/STRING('IJ'))
L/CLOSE/MEMBER(L/TOBJ(OPF))
L/OPEN/MEMBER(L/TOBJ(OPF),ADD)
L/ASSIGN(L/TOBJ(M.A),LC/STRING('CD'))
L/ASSIGN(L/TOBJ(M.B),LC/STRING('LM'))
L/CLOSE/MEMBER(L/TOBJ(OPF))The ADD suboperation makes the member you just added the current member 🦋, so you don't need any L/WHICH/MEMBER calls in this sequence.
To end the sequence of listops:
L/END/LISTOP(L/TOBJ(OPF)) _________________
| _____________ |
| | F | |
| |_____________| |
| NAME |
| _____________ |
| | ____|_|_________\
| |_____________| | /
| DESCRIPTION |
| _____________ |
| | | |
| |_________|___| |
| VALUE | |
|___________|_____|
OBJECT |
|
_______________V______________
| __________________________ |
| | _________ _________ | |
| | | | | | | |
| | | "AB" | | "CD" | | |
| | |_________| |_________| | |
| |__________________________| |
| __________________________ |
| | _________ _________ | |
| | | | | | | |
| | | "EF" | | "GH" | | |
| | |_________| |_________| | |
| |__________________________| |
| __________________________ |
| | _________ _________ | |
| | | | | | | |
| | | "AB" | | "IJ" | | |
| | |_________| |_________| | |
| |__________________________| |
| __________________________ |
| | _________ _________ | |
| | | | | | | |
| | | "CD" | | "LM" | | |
| | |_________| |_________| | |
| |__________________________| |
|______________________________|
VALUE
Figure 4-20
After L/END/LISTOPNext up, a slightly spicier challenge 🔥 Build calls that create a LIST named G with the same description as F, then copy every member of F whose A is 'AB' into G. 💅
We need to create G first, plus an OPD and an object to represent the current member:
L/CREATE("STAR",LD/LIST(G,
LD/STRUCT(R,
LD/STRING(A,STRING,2),
LD/STRING(B,STRING,2)))
L/CREATE(L/TOBJ(TOP/LEVEL),LD/OPD(OPG))
L/CREATE(L/TOBJ(TOP/LEVEL) ,LD/STRUCT(GM,
LD/STRING(A,STRING,2),
LD/STRING(B,STRING,2)))Now we start two listop sequences, one on G and one on F:
L/BEGIN/LISTOP(L/POBJ(F),L/TOBJ(M),
L/TOBJ(OPF),GET)
L/BEGIN/LISTOP(L/POBJ(G),L/TOBJ(GM),
L/TOBJ(OPG),ADD)
L/WHICH/MEMBER(L/TOBJ(OPF),FIRST)
L/WHICH/MEMBER(L/TOBJ(OPG),LAST)We're going to walk through the members of F. Whenever the current member has A equal to 'AB', we add a member to G and copy the current member of F's values into it. If the current member doesn't match, we leave it alone. 🧚
First, we need a loop that runs until we hit the end of F:
L/TILL(L/END/OF/LIST(L/TOBJ(OPF)),x)Whatever replaces "x" in this call runs over and over until the end/of/list flag gets set in OPF. 🌸
Here's the catch: L/TILL wants exactly one function call in that spot, but we need to run "x" once per member of F and do several listops every time. The fix is L/CF, the compound-function call:
L/TILL(L/END/OF/LIST(L/TOBJ(OPF)),L/CF(y))Now "y" can be a whole sequence of function calls. 👑
Each time around, we need a fresh member of F to work on. We start out set up to get the first one, so the sequence looks like this:
L/CF( L/OPEN/MEMBER(L/TOBJ(OPF),GET),
z
L/CLOSE/MEMBER(L/TOBJ(OPF)),
L/WHICH/MEMBER(L/TOBJ(OPF),NEXT) )That compound function opens the current member of F, does something with it (that's the "z" bit), closes it, and then asks for the next member. 💖
For "z", we want a call that checks the A field of the current member of F, then either does nothing or adds a member to G and copies F's values over. If "w" stands for the add-and-copy action, it looks like this:
L/IF(L/EQUAL(L/TOBJ(M.A),LC/STRING('AB')),w)A good way to write "add a member and copy the values" is:
L/CF(L/OPEN/MEMBER(L/TOBJ(OPG),ADD),
L/ASSIGN(L/TOBJ(GM.A),L/TOBJ(M.A)),
L/ASSIGN(L/TOBJ(GM.B),L/TOBJ(M.B)),
L/CLOSE/MEMBER(L/TOBJ(OPG))This is pretty much the same pattern as the last example, so it should feel familiar 🫶
Putting it all together:
L/TILL(L/END/OF/LIST(L/TOBJ(OPF)),
L/CF( L/OPEN/MEMBER(L/TOBJ(OPF),GET),
L/IF(L/EQUAL(L/TOBJ(A),LC/STRING('AB')),
L/CF( L/OPEN/MEMBER(L/TOBJ(OPG),ADD),
L/ASSIGN(L/TOBJ(GM.A),L/TOBJ(M.A)),
L/ASSIGN(L/TOBJ(GM.B),L/TOBJ(M.B)),
L/CLOSE/MEMBER/L/TOBJ(OPG)) ) )
L/CLOSE/MEMBER(L/TOBJ(OPF)),
L/WHICH/MEMBER(L/TOBJ(OPF),NEXT) ) )To wrap the whole operation up, we run:
L/LISTOP/END(L/TOBJ(OPG))
L/LISTOP/END(L/TOBJ(OPF))The result is a LIST G whose first member has value ('AB','CD'), and whose second member has value ('AB','IJ'). With a few tweaks to the example above, you can run quite a few LIST operations, bestie 🎀✨
4.11. Higher level functions 🌸
okay so these primitive i/functions are useful, but nobody is gonna be typing datalanguage at this super low level 😭. we want the primitives out there so users can handle the exceptional cases, and so they can build their own high-level functions for the atypical stuff 💅. ordinarily, people ought to work at least at the level of the construction below (which is legal in the real datalanguage that's currently implemented 🧚):
FOR G.R,F.R WITH A EQ 'AB'
G.R=F.R
ENDthis relatively concise expression gets the same result as the elaborate construction of i/functions at the end of the previous section, just way less ugly 🦋. we could define i/functions that are super similar to the semantic functions used in the running software, and write the request above as:
L/FOR(L/POBJ(G),R
L/POBJ(F),R,L/WITH(L/EQUAL(L/TOBJ(A),
LC/STRING('AB')))the differences between the i/function call and the datalanguage request above it are mostly syntactic, so it's giving the same energy 💖.
in designing functions like L/FOR and L/WITH, the big problems are all about picking the right restrictions 👑. one cannot have all the generality available at the primitive level (lol, no). some important choices for these particular functions are:
- handling multiple inputs and outputs,
- when FORs are nested, how outer FORs restrict the options available to inner FORs,
- generality of selection functions (may then in turn generate FORs?),
- options with regard to where processing should start (are we updating, replacing or appending to the output list(s)?).
finally, this problem ties into the bigger problem of dealing with sets, which are a generalization of the idea of a collection of members in a LIST having common properties ⭐. FOR is only one of many operators that can input sets, so it's not the whole story 🫶.
4.12. Conclusion 🍼
the present model is still baby-stage, but it already has enough primitives and data types to define and generally manipulate hierarchical data structures ✨. common data management stuff, like retrieval by content and selective update, can be expressed 💅.
using this model to develop these primitives gave us precise, well-defined and internally consistent specifications for language elements and processing functions 🔥. operating in the laboratory environment the model provides seems to be a substantial benefit 🧪.
5. Further Work 🌈
in this section, we look at what's been accomplished so far in the design and what work is still left before this design iteration of datalanguage is complete 🧚.
5.1. A Review 👑
most important, we feel we've delineated the problems and presented the broad outlines of a solution for developing a language for the datacomputer system, and that's iconic 😭. the key elements of our approach are:
- the primacy of data description in capturing all the aspects of the data,
- the separation of logical and physical characteristics of data description,
- the ability of users to define different views of the same data,
- the ability to associate functions with different uses of data items,
- an attempt to capture common aspects of data at every possible level,
- and the ability of users to communicate with the datacomputer at as high a level as their application permits.
5.2. Topics for Further Research 🔍
although more work is needed in general to turn out a finished design for datalanguage, we can single out certain issues that especially need investigating 🙈.
so far, only hierarchal data structures (i.e. those that can be modeled by physical containment) have been developed to any real extent. we also intend to investigate and provide other types of data structures. we're confident our language framework doesn't make assumptions that would block those additions 🦋.
our current work on access regulation centers on using multiple descriptions for data. we need to do more work on both the technical and administrative aspects of access regulation. problems of encrypting data for both transmission and storage will also be investigated 🔐.
another issue needing further research is the protocol requirement for process interaction with the datacomputer 💜.
separation of the description into independent modules needs further research too. in particular, we need to look into work that's already been done on separate specifications of logical descriptions, physical descriptions, and mappings between the two 🍓.
5.3. Datalanguage Syntax 🎀
we haven't proposed a syntax for the datalanguage yet 🙈. certainly the hardest parts of the problem have been the semantic and pragmatic issues. we're confident various syntactic forms can be chosen and implemented without excessive difficulty 💅.
it may be best to develop different syntactic forms for different types of users, or even for the various subparts of the language itself. as discussed in section 2, the user syntax for the datacomputer is supposed to be at a low level. it should be easy for programs to generate datalanguage requests in this syntax 🤖✨.
5.4. Further Work on the Datalanguage Model 🧚
the model provides an excellent foundation for building up a language with the facilities described in section 3, but there's still a lot to do 💖.
for a while, emphasis will be on sets, high-level operators, language extension and data description 👑.
we expect to model sets as a new datatype, whose value is ordinarily shared with other objects. some further work on binding and sharing of values is needed to support this 🫶.
sets can be regarded as a special case of generalized relations, which will come somewhat later ⭐.
high-level operators such as FOR will be constructed from the existing primitives, and will eventually be defined to have one effect but several possible expansions. the expansion will depend on the representation of the data and the presence of auxiliary structures 🌸.
alternate expansions will be possible when the data description has been broken up into its various modules. this, also, requires some further research 🔍.
we feel the language extension problem is much more easily attacked in the environment provided by the model datacomputer. in particular, we expect the laboratory environment to be helpful in evaluating the complex interactions and pervasive effects of the operators that extend the language 🧪.
data description work in the near term will focus on the isolation of attributes, the representation of variable structure in description, the description of descriptions, and the development of a sufficient set of builtin data types 🌈.
later, we expect to model the semantics of pointers as a datatype, once the representation of the pointer and the semantics of the address space it points into are specified in the description of the pointer 🦋.
a large number of lower-level issues will be attacked, including some of the problems discovered in the modeling to date. some of these are pointed out in the discussions in section 4 🙈.
5.5. Applications Support 💅
the datalanguage we're designing is intended to provide services to sub-systems solving a broad class of problems related to data management. examples of such sub-systems are: report generators, online query systems for non-programmers, document-handling systems, transaction processing systems, real-time data collection systems, and library and bibliographic systems. there are many more 📚.
the idea is that such systems will run on other machines, reference or store data at the datacomputer, and make heavy use of datalanguage. such a system wouldn't be written entirely in datalanguage, but a large component of its function would be expressed in datalanguage requests. some controlling module would build the requests and perform the non-datalanguage functions 🧩.
we have experience with such applications in other environments, and we talk to potential users, but it will take some work to determine that our language is actually adequate for them. we aren't attacking the problem of building a human-oriented online query system directly; we're trying to provide the tools that will make it easy to build one 🛠️. there's a definite need to analyze whether the tools are likely to be good enough. the ultimate test will be actual use, but we want to filter out as many problems as we can before implementation 🔥.
an important part of supporting applications is that the using programs will frequently be written in high-level languages such as FORTRAN, COBOL or PL/1. we will want to investigate how well datalanguage can support such users while the design is taking shape 💜.
5.6. Future Plans 🚀
this paper has laid the foundations for a new design of datalanguage. section 3 provides an outline for a datalanguage design, which will be filled in during the coming months. following the issue of a detailed specification, we anticipate extensive review, revisions, and incorporation into the implementation plans. implementation will occur in stages, compatible with the established plans for development of datacomputer service and data management capabilities 🌸✨.
[ This RFC was put into machine readable form for entry ]
[ into the online RFC archives by Alex McKenzie with ]
[ support from GTE, formerly BBN Corp. 1/2000 ]📦 where this page comes from
- ietf: rfc610 1973-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms