RFC 369

Evaluation of ARPANET services January-March, 1972

📅 published July 1972, from ietf

Evaluation of ARPANET Services ✨

January through March, 1972

okay bestie, let's get into RFC 369 🦋 It was written by J. Pickens from the UCSB Computer Systems Laboratory on 25 July 1972, filed under the Network Working Group with NIC number 6801. The header doesn't name a category or status, and it doesn't obsolete or update anything. main character energy from the very first line 💅

Abstract 📖

okay so this one opens by pointing back at RFC #302, Exercising the ARPANET 🌸 That earlier piece described a group at UCSB that was set up to poke around the network's resources. The goals were to build problem solving skills and, while doing that, give useful feedback to the nodes they explored. This report sums up what the group went through and found, and it suggests network refinements that could make users happier 🫶

The group ran into a ton of unexpected problems, ngl. The biggest ones were software heterogeneity (every site doing things its own way) and inadequate documentation 😭

From all that first hand experience, the group came up with criteria for how easy network resources are to use ✨ The report lays those criteria out, plus ideas for better documentation, making better use of what already exists, and a plea for people to actually use the inter-personal communication tools 💜 Individual sites got graded on reliability, response, and friendliness, and there are comments about specific sites to help new users find their way 🧚

Even though the first nine weeks were rough, the group still learned enough about ARPANET resources to write useful software 💖 They built programs for automatic login, file transfer, and interprocess communication, and those programs are in use now 🎀

Table of Contents 🗂️

BACKGROUND
   Approach.......................................  2
   Goals..........................................  2
THE SURVEY
   Extent and Duration............................  3
   Statistical Results............................  3
CRITIQUE OF ARPANET SERVICES
   A Site Measurement Parameter, "Friendliness"...  4
   Software Critique..............................  5
   Community Spirit...............................  5
   Economics......................................  6

SUGGESTIONS FOR IMPROVEMENT
   Software.......................................  6
   Community Spirit...............................  7
CONCLUSION........................................  8
APPENDIX A
   Sample of Survey Questionnaire................   9
APPENDIX B
   Grades and Comments for Specifics Sites.......  10

Background 🌷

Approach 🧑‍🎓

the test group was put together from a bunch of Electrical Engineering graduate students in Computer Science 🎓 A lot of them already had solid experience with high level languages and time sharing systems, like the Dartmouth BASIC and UCSB mathematical graphics systems 💅 BUT nobody had actually used ARPANET, and few knew what resources it even offered 🙈 After two weeks of presentations from Jim White and Roland Bryan, the group was let loose to experiment freely 🦋

The group was super hyped at first. Each team managed to find and decode the login procedures for different nodes and started figuring out what they could do 🔥 Half of the weekly seminar time was basically group chat for sharing what everyone had learned 💬 But the interest kinda dipped as the quarter went on, because of poor network site reliability, few active nodes, and documentation that was hard to find 😩 Only five out of fourteen students were still active after the first quarter 💀

Goals 🎯

The main goal was to learn how to use network resources and evaluate them ✨ The group decided to be fair but direct when judging each site, UCSB included 💅 Since everyone's networking experience was pretty low at the start, the evaluation criteria were mostly based on gut feelings 🙈

At the end of the first quarter, the students got a questionnaire (there's a sample in Appendix A) 📝 The overall group response is summarized below, and the data for individual sites is in Appendix B 🌈 Some of the questions were:

Estimate percentage of time spent in various trouble states
Estimate the mean time to failure
Describe personal experience with the network
Suggest improvements
Grade the investigated nodes on the factors of reliability, response, and friendliness

The Survey 🔍

Extent and Duration ⏳

During the main stretch of work (January to March, 1972), not many nodes were active 🥺 So the experiments were mostly focused on UCSB, BBN-TENEX, MIT-MULTICS, and SRI-ARC 💜 HARV-10, UCLA-NMC, and UCLA-CCN got a little bit of poking around too 🦋 The remaining sites were either inactive or unreachable because there wasn't any documentation for them 😭

The activities included:

Game playing (e.g., chess, life, and doctor at BBN-TENEX) 🎮
Text and file manipulation (e.g., COL, NLS, TECO) 📝
Inter-personal communication (LINK and SNDMSG) 💌
On line compilation (e.g., TENEX FORTRAN, MULTICS PL/1) 🖥️

Statistical Results 📊

Figure 1 sums up how the group answered the questionnaire after nine weeks with ARPANET 🌸 Individual exposure ranged from ten to sixty hours, and twelve students responded 🧚 Each survey item is shown as a group average (sum/12), along with a low and a high value so you can see the range of responses ✨ The questions were a little fuzzy, because they didn't tell apart node inactivity from local NCP inactivity 🙈 Some numbers might also reflect individual students not fully understanding local and foreign procedures 😅 Still, the data is super interesting as a look at what uninitiated users go through 💖

Figure 1 📈

Survey Item                                Average    Low    High

% of time unable to log in any site         12,4%     2%     25%
% of time unable to log into desired site   35.7      20     75
% of time foreign site suddenly crashes     13        5      50
% of time local site suddenly crashes       12.5      5      25
% of time trouble free operation            35        0      80
Approximate mean-time-between-failure       1h       5 min   2 hrs

TOTAL TIME INVESTED                         32.3hrs  10 hrs  60 hrs

First up, if you look at the whole ARPANET complex, nobody got the mean-time-between-failure above two hours! 😱 Second, the average for "trouble free" operation was 35%, which is honestly not something you could live with as a regular user 💀 To be fair though, some sites were way more "trouble free" than others, and people tend to judge that by their own level of skill and experience 🤷‍♀️ That's why the high was 80% and the low was 0% ✨

Critique of ARPANET Services 🧐

A Site Measurement Parameter, Friendliness 🫶

The group talked a LOT about "friendliness", especially for on-line systems 💬 Based on what they actually experienced on the network, here's their definition 🌈

Friendliness is:

Concise, complete, and available documentation.
Easy system usage (e.g., minimum numbers of keys for login system and job status readily available).
Easy to reach help both on-line people and on-line files.
No messages overkill (as sometimes unexpectedly occurs during login).
Reasonable reliability and response time.
Concise, but informative error diagnostics.

The reader can probably think of more criteria too 💖 But these were the standout points of friendliness that came directly out of the group's experience 🦋

Software Critique 💻

The first experiments focused on login procedures, canned scenarios (e.g., Abhay K. Bhushan's ARPANET scenario, RFC #254), game playing, and inter-personal communication 🎮 As things went on, people started trying to solve actual problems at different nodes 🔥 One student, for example, programmed a Newton-Raphson root finder in PL/1 at MIT-MULTICS a blackbody problem in FORTRAN at BBN-TENEX and MIT-MULTICS, and in PL/1 at MIT-MULTICS; and a Discrete Fourier Transform in BASIC at BBN-TENEX 🤯 The group concluded that small problems could be written in a half hour, entered and edited in fifteen minutes, and debugged in another fifteen minutes ⏱️ For small problems, the current ARPANET software resources were honestly pretty adequate 🙌
By far the most annoying thing was getting adequate documentation 😤 The resource notebook was interesting, but it was of limited use 📚
Information about each node's NCP, which was requested in February, 1972, is still unavailable 🥲
Procedures varied a LOT when doing similar tasks on different nodes 🙃 Look at the huge range of text editors, with their own file naming, editing, and manipulation commands (TENEX, TECO, COL, NLS...) 📝 Then there's the whole mess of compilation, load, and execute procedures (RJE for UCSB edit, save, compile, save, load, execute for TENEX systems) 🌀 Even more all over the place are the "executive level" commands (TENEX's "Control-C", UCLA-NMC's "X", UCSB's "RESET" ... all of which return to the "top-lvel") 💀 Software heterogeneity is a real stumbling block for users 🚧
People at large nodes have a hard time finding problems that need solving outside their local environment 🏠 With UCSB's mathematical graphics on-line system and direct access to batch, the group felt a little apprehensive 😬 They spent hours on the network solving problems that could've been solved in minutes locally ⏳

Community Spirit 🤝

People sometimes got the impression (hopefully wrongly) that some researchers in the ARPA community had little desire to consult or help 😬 On the other hand, others bent over backwards to give assistance 🫶 The group had hoped for a more consistent response 🥺
It was hard to figure out who was responsible for developing resources 🔍 The seminar group felt that responsibility being spread out so completely took away the incentive to locate, document, and create useful network resources 💔

Economics 💸

Network economics, both above and below the communications level, are a big problem for users 🙈 For example, if distributed computing is allowed, then distributed billing is a necessity 💳 It's super frustrating to watch accounts randomly die at different nodes and then spend weeks trying to sort out the money stuff 😭 That happened with a site that (a) randomly changed passwords and then (b) got rid of its free account 🫠 There's also a problem with double connect charges, like $4.00 per hour at UCSB to sign on to BBN-TENEX at $8.00 per hour, which totals to $12.00 per hour! 🤑

Suggestions for Improvement 🚀

In spite of all the difficulties and frustrations, the class was really impressed by the potential of ARPANET 🌟 They came up with several suggestions for improvement, and part 2 picks them up right here 🦋

Software ✨

okay bestie, the first big idea is that working groups should be put together to tackle problems that need a significant chunk of the network's resources 💅

Second, the ARPANET is already a massive resource, even though TELNET is the only operational protocol so far 🌸 So more effort should go into actually using what we already have. Here are two examples that are giving main character energy:

a) By combining UCSB's OLS and UCSB's TELNET, user programs were created to sign on automatically to the various sites. So a network user only needs to know the sign-on procedure for UCSB. The settings for local/remote echo, character/line at a time, upper/lower case, etc. are all handled automatically by the pre-written user programs 🎀
b) A group of students combined the TELNET PROTOCOL, PL/1 subroutine calls to the UCSB NCP, and 360 O/S multi-programming to build a batch-fed command language in PL/1 that talks to foreign sites over telnet. It has been used successfully to look into file transfer (NIC files are regularly copied onto 8-1/2 x 11" white printer paper, and cards will soon be transferred to I4-TENEX), interprocess communication (a program was started at BBN-TENEX to be used as a subroutine locally, and plans exist to start and monitor a chess game between BBN-TENEX and SU-AI), and data transfer (pre-formatted data files have been transferred from UCLA-NMC to UCSB, and UCLA-NMC will soon make survey and measurement data available via TELNET PROTOCOL and through direct ICP!). Moe details of this program will be available in a future report 🧚

Third, documentation. Every site, and also every network-wide function like the FORTRAN compilers, should have a self-sufficient mini-user-manual (MINIMAN) 📖 A MINIMAN is kind of similar to the resource notebook, but it's way more focused on helping the user actually run stuff. A site-dependent MINIMAN would contain:

Sign on procedure
Simple file manipulation and editing commands
Compilation and execution instructions
TELNET access
Brief (!) summary of programs and subroutines
Direction on how to get help

Overall documentation of hardware, software and human resources should be more complete 💜 Maybe a documentation questionnaire should go out to authors of network programs, including the authors of Network Control Programs. Merging the questionnaire info with the Resource Notebook would make it way easier to build a resource-location cross referenced index. That kind of index, maybe on-line, would help network users find both software and hardware. Whatever the final scheme, more planning is needed to win the user versus documentation battle 😭 The recent effort in this direction by Marshall D. Abrams, "Serving Remote Users on the ARPANET" (NIC 10606 RFC #364), is perfectly timed and should be thoroughly considered 🫶

Last but not least, high level subroutine calls to each NCP, like the ones UCSB offers, should be universally available 👑

Community Spirit 🦋

First, networks have great but unexploited potential for interpersonal communication 💖 Tools like NIC's JOURNAL, NLS TENEX's SENDMSG, LINK, and UCLA-NMC'S S_.MSG:C are only used today by the proficient few, but they should be used regularly by everyone. The group noticed two symptoms of the current state of network communications: most procedural info was shared verbally in class, and lots of problems finding documentation got solved by the ultimate last resort, the telephone 📞 Better communication will boost cooperation on joint projects, obviously 🌈

Second, names and interests of programmers and researchers who are willing to cooperate on joint projects should be kept track of, along with "blue sky" lists of software projects ✨

Third, a network NEWS and NOTES should be published to inform and advise participants about new resources and procedural changes. Just be careful to keep it concise (aka avoid "message over-kill" 🙈). Maybe a one page flier published weekly would do the trick 🍓

Fourth, a network consulting center should be created, maybe at the existing NIC, specializing in non-partisan matching of network users to network resources 🎀

Fifth, the network has a strong potential in Computer Science education 🧠 Being exposed to lots of different computer systems helps students and users avoid the narrowness of experience and opinion that sometimes exists in centers of learning and computing. The TIP user is probably the most benefited here, because for little investment in local resources, many styles of systems are right at their "finger-tips" 💅 Even for service nodes, the network is an inexpensive extension to local educational resources. Current efforts to tap the educational value of the ARPANET should be encouraged and extended 🌸

CONCLUSION 🎓

Existing site surveys measure and evaluate the performance of IMP hardware, host hardware, and host NCP programs, but very little has been done to evaluate software performance 😬 The UCSB EE 210 graduate students made a primitive first pass at evaluating network resources between January and March 1972. Out of that effort came definitions and criteria that would be useful to other individuals or agencies building evaluation schemes at the USER protocol level. So yeah, the hope is that this report is useful 🫶

APPENDIX A - Sample Student Questionnaire 📝

ARPANET

   Grade Given:  A=Excellent                 Evaluation by:
                 F=Bad

   -------------------------------------------------------------------
   SITE | RELIABILITY| RESPONSE | FRIENDLINESS | # HOURS  | COMMENTS |
        |            |          |              |   USED   |          |
   -----|------------|----------|--------------|----------|----------|
        |            |          |              |          |          |
        |            |          |              |          |          |

                            ARPANET Evaluation

   -- Indicate % of your sessions which were in the following categories:

              %               State
         +--------+-------------------------------------------+
         |        |  Unable to Log in to any site.            |
         |--------|-------------------------------------------|
         |        |  Unable to Log in to Desired site.        |
         |--------|-------------------------------------------|
         |        |  Foreign site suddenly crashes.           |
         |--------|-------------------------------------------|
         |        |  Local site crashes.                      |
         |--------|-------------------------------------------|
         |        |  Trouble free operation.                  |
         |--------|-------------------------------------------|
         |        |  Other                                    |
         +--------+-------------------------------------------+

   -- Considering the performance of the local host, communication
      network, and remote hosts, estimate the mean time to failure of
      ARPANET:
         Mean-Time-Between-Failure=___________

   -- What was your total time invested in the ARPANET this quarter?
         Total Time Invested=___________

   -- Describe your overall experience with the ARPANET (e.g., rise and
      fall of personal interest factors involved, etc.).

   -- What suggestions for changes or improvements or new capabilities
      do you have to make to ARPANET hosts?

      (Use back side or other paper for these questions if necessary)

APPENDIX B - Specific Sites, Grades and Comments 📊

The following grades and comments are based on the two to four most representative questionnaire responses for each site. Reliability, Response, and Friendliness are averaged grades and reflect subjective criticism. Total Invested time is the sum total of the experimentation times reported by individual respondents. It is hoped that future evaluations might be more specific and complete than the current efforts, yet the value of these initial efforts should not be underestimated 💖

Grades:

      A=Excellent

      F=Bad

                                                    Total Time
   Site        Reliability  Response  Friendliness   Invested
   --------------------------------------------------------------
   BBN-TENEX       A            A         A             71 hours
   UCSB            B            B+        B-            36
   SRI-ARC         B            B         A             75
   HARV-10         C            A-        B             14
   UCLA-NMC        C-           C         D             14
   MIT-MULTICS     C-           D         C+            82
   --------------------------------------------------------------

Group Comments

      Site:  BBN-TENEX
         Very popular site
         Doctor, life and chess are stimulating and easy to use games
         Operators are very helpful
         Account problems kept site from being useful
         BASIC is well-written and easy to use
         FORTRAN is difficult to use because of the many steps to
         create-compile-execute.

      Site:  UCSB
         There are many problems with old key boards
         TELNET diagnostics are poor
         Online help files are sorely lacking
         Graphics are necessary for full utility
         Operator would not reload NCP when down
         List of TELNET site names are not current or complete

      Site:  SRI-ARC
         Good documentation exists on NLS specifics, but general
           overview is lacking
         Inter-console link is convenient and often used.
         NLS-JOURNAL is useful but requires significant training
         Online perusal is difficult at terminals with small display faces.

      Site:  HARV-10
         Operator is readily available
         FORTRAN is straight forward
         Easy to use editor
         Couldn't get operator to put BASIC on.

      Site:  UCLA-NMC
         Self-explanatory ABACUS program is not self-explanatory
         System often disappears
         Hard to get past LOG ON* without TIMEOUT GOODBYE
         Message system is well organized.

      Site:  UCLA-CCN
         Always up, but nothing can be done (HELP is not supported)
         When RJS is executed, there is no response until correct signon
         procedure is entered (spurious death indication).

      Site:  MIT-MULTICS
         Response is very slow
         Automatic logout of autonomous user is excruciatingly painful
         Text editor is very easy and helpful
         PL/1 and FORTRAN are easy to use.

[This RFC was put into machine readable form for entry] [into the online RFC archives by H�l�ne Morin, Viag�nie 12/99]

📦 where this page comes from

ietf: rfc369 1972-07
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms