RFC 565

Storing Network Survey Data at the Datacomputer ✨

Okay bestie, the title block first! 🎀 This RFC is RFC 565, filed under the Network Working Group on 28 August 1973 📅 by D. Cantor of the Computer Corporation of America (NIC 18777). The header doesn't list a category, and it doesn't obsolete or update any other RFC. 🦋

---

In November, 1972, Computer Corporation of America (CCA) and the Programming Technology Division of the Dynamics Modeling System (DMS) at M.I.T.'s Project MAC started planning to send information about the behavior of ARPA network hosts to CCA's datacomputer [1] 💅. That information was collected by DMS's program SURVEY [2]. It was going to be stored at the datacomputer and pulled back out by an interactive program that talks to the datacomputer from DMS's PDP-10. 🌈

One goal of this joint project was to let DMS keep everything SURVEY collects 😭. SURVEY had been running since late 1971, but it only saved a short daily summary and threw away potentially useful details. The second goal was to find and fix the shortcomings in the interface between CCA's datacomputer and a program running on a remote host. 🔥

The project was finished last month, and the programs described in this document have been running successfully with the datacomputer since July 10. ✨

Part 1 below covers SURVEY's output. Part 2 covers a program that pulls portions of that output back out of the datacomputer. 🧚

---

Part 1: The Survey Database 🌸

Every twenty minutes, SURVEY wakes up from DMS's PDP-10 and does the initial connection protocol with the logger socket (socket 1) of each of the 28 network hosts. 💜

SURVEY records a date, time, host, status, and response time for each host. A host can be in one of these states:

• undetermined or not surveyed
• disconnect from the network or dead
• network control program not responding
• ICP to logger aborted by the host
• ICP to logger timed out by SURVEY after 20 seconds
• logger available and responding within 20 seconds

SURVEY records response times in tenths of seconds. ⏱️

Once the data for all 28 hosts has been gathered, SURVEY sends it to CCA's datacomputer. If the datacomputer can't accept the information for some reason, it gets held at DMS and sent again another time. 🫶

The datacomputer's survey database is inverted by host, status, month, and year. Basically, the datacomputer keeps several indices pointing to the records of one attempt to establish a full duplex connection to one host's logger. It keeps one index for each host, one for each status, one for each month, and one for each year. 🍓 It can select records matching boolean expressions by doing boolean operations on the inversion, without ever looking at the data itself. That's what makes the survey retrieval program below so fast when it talks to the survey database at the datacomputer. 🔥

SURVEY writes the record of each attempt to access one host in 17 ASCII characters. So one survey's record takes up 17 * 28 = 476 characters, and each day the datacomputer receives 3 * 24 * 476 = 34,272 characters from DMS. 😳

---

Part 2: Retrieving Survey Data 👑

A program called SURRET, written at DMS in the language MUDDLE, lets you selectively pull material out of the survey database stored at the datacomputer [3]. Its user can specify values, groups of values, or, where it makes sense, upper and lower bounds for values of each of five fields: host name, date, time, response time, and host status. You can also ask for all five fields, or any subset of them. Here's a sample interaction with SURRET. 💖

 <HOST (CASE-10)>$
"OK"
<DATE (AUG 5 73)>$
"OK"
<TIME (BETWEEN 2000 2400)>$
"OK"
<REQ ((TIME STATUS RESTIME))>$
;J205 10-08-73 1557:20 RHRUN: SUCCESSFUL COMPILATION
.1241 10-08-73 1557:21 OCSOP: (DEFAULT) OUTPUT PORT OPENED

TIME           STATUS             R.T.(1/10 SEC)
2004    LOGGER RESPONDING (UP)        019
2024    LOGGER RESPONDING (UP)        024
2044    LOGGER RESPONDING (UP)        021
2104    LOGGER RESPONDING (UP)        016
2124    LOGGER RESPONDING (UP)        046
2144    LOGGER RESPONDING (UP)        018
2204    LOGGER RESPONDING (UP)        017
2224    LOGGER RESPONDING (UP)        017
2244    LOGGER RESPONDING (UP)        023
2304    LOGGER RESPONDING (UP)        015
2324    LOGGER RESPONDING (UP)        016
2344    LOGGER RESPONDING (UP)        015
"END OF DATCOMPUTER OUTPUT"

The angle brackets, whatever they wrap, and the '$' (ESC or altmode) were typed by a person using SURRET. Everything else was typed by the system. The phrases in quotation marks are, effectively, SURRET prompts. The status messages starting with ';' and '.' were generated by the datacomputer. SURRET formatted the column headings and table using figures it retrieved from the datacomputer. 🎀

SURRET generates datalanguage, sends it to the datacomputer, and processes system diagnostics and data the datacomputer sends back to it. The datalanguage generated for the request above was:

FOR |SURVEY.LOGTRY, SURVEY.LOGTRY WITH
  ((YEAR EQ '73' AND MONTH EQ '08' AND DAY EQ '05')
  AND (HRMIN GE '2000' AND HRMIN LE '2400')
  AND (HOST EQ '013'))
HRMIN=HRMIN ;  STATUS=STATUS  ;  RESTIME=RESTIME  ;
END;

The field names in the datalanguage were entered with file descriptors before the first data was loaded. 🧾

You can ask SURRET to retrieve new data by changing the values of any number of fields and issuing a new REQ (request). The <state> command shows the current values for the five possible retrieval criteria. So:

 <HOST (USC-44)>$
"OK"
<STATE>$
!HOST: (USC-44)STATUS: () RESTIME: () DATE: (AUG 5 73)
  TIME: (BETWEEN 2000 2400)!

<REQ ((TIME STATUS RESTIME))>$
;J205 10-08-73 1610:08 RHRUN: SUCCESSFUL COMPILATION
.1241 10-08-73 1610:09 OCSOP: (DEFAULT) OUTPUT PORT OPENED

TIME           STATUS             R.T.(1/10 SEC)
2004    LOGGER RESPONDING (UP)        020
2024    LOGGER RESPONDING (UP)        008
2044    LOGGER RESPONDING (UP)        008
2104    LOGGER RESPONDING (UP)        009
2124    LOGGER NOT RESPONDING (LNR)   000
2144    LOGGER NOT AVAILABLE(DEAD)    000
2204    NCP NOT RESPONDING (NNR)      000
2224    LOGGER NOT RESPONDING (LNR)   000
2244    LOGGER NOT AVAILABLE (DEAD)   000
2304    LOGGER NOT AVAILABLE (DEAD    000
2324    NCP NOT RESPONDING (NNR)      000
2344    LOGGER RESPONDING (UP)        007
"END OF DATCOMPUTER OUTPUT"

We could have retrieved all of the output above with:

      <HOST (CASE-10 OR USC-44)>

Also:

      <HOST (CASE-10 CCA OR USC-44)>

would make SURRET access the database twice: once for information about Case-10, and then a second time for information about the remaining two hosts. 🤓

Detailed Survey data from July 10, 1973 onward is available directly from the datacomputer or through SURRET. If you want to use the datacomputer directly, you can get the relevant documents through the NIC, or by contacting Dale Stern at CCA (617-491-3670). 📞

---

Endnotes 📚

[1] An overview of the data computer is given in Thomas Marill, The Datacomputer, 18 Oct '71, 7pp. (NIC 7979). A detailed study of the programming language for addressing the datacomputer is found in Computer Corporation of America, Datacomputer Project Working Paper No. 3, Datalanguage, 29 Oct '71, 78 pp. (NIC 8208). The current status of the language is reviewed in Richard Winter, Specifications for Datalanguage, Version 0/9, 6 Jun '73, 36 pp. (NIC 16446). A user's manual for version 0/9, will be released by CCA in September, 1973.

[2] SURVEY is described in Abhay Bhushan, A Report on the Survey Project, 22 June '73 (NIC 17375).

[3] A detailed discussion of SURRET is found in Safwan Bengelloun, MUDDLE Survey Data Retrieval Programs, an internal DMS memo of 14 June, '73. Our purpose here is to describe enough of the program's syntax and structure to show how it interacts with the datacomputer.

[ This RFC was put into machine readable form for entry into the online RFC archives by Via Genie 08/00 ] 🌈