RFC 392

Measurement of Host Costs for Transmitting Network Data 💖

• Network Working Group 🎀
• RFC: 392
• NIC: 11584
• Authors: G. Hicks and B. Wessler, both from Utah 🦋
• Date: 20 September 1972 🌸
• Category/status: not stated in this document
• Obsoletes/Updates: neither is stated in this document

Background for the UTAH Timing Experiments ✨

okay so, since October 1971 we at the University of Utah have been running really big compute bound jobs every day 💻 these jobs ran for many cpu hours just to get partial results, and they used resources that could honestly be better obtained somewhere else. we felt that since these processes were being treated as batch jobs, they should be run on a batch machine 🧚

so to meet the needs of these "batch" users, in March of this year, we developed a program[1] to use the Remote Job Service System (RJS) at UCLA-CCN. RJS at UCLA runs on an IBM 360/91 🔥

some examples of these jobs were (and still are!) 😭

• (a) Algebraic simplification (using LISP and REDUCE)
• (b) Applications of partial differential equation solving
• (c) Waveform processing (both audio and video)

the jobs run on the 91 were mostly small data decks getting submitted to RJS and massive print files getting retrieved. with one exception: the waveform processing group needed, from time to time, to store large data files at UCLA for later processing. when this group did their processing, they retrieved very large punch files that were later displayed or listened to here 🎧

when the program went live in late march, and everyone started using it as a matter of course, users complained that it page faulted a lot 😩 we restructured the program so that the parts that got used often did not cross page boundaries 💅

the protocol with RJS at UCLA requires that all programs and data sent on the data connection be blocked[2]. that means we simulate records and blocks with special headers. we found this to be another problem because of the computation and core space involved. the computation took an appreciable amount of time and core space. because of our real core size, we were being charged an excessive amount due to page faulting, and that page faulting also lowered our real-time transmission rate enough that we felt the transmitting and receiving portions of the program needed a re-write 🛠️ to get the best service from the system, we optimized these portions so that each one occupied a little over half of a page. since we now had so few pages in core at any one time, the TENEX scheduler could give the program preference over larger working set jobs. (as an aside, because of our limited core, we wrote a small (one and one half pages) editor to provide an interactive text editing service 📝)

the mechanism to access the network under TENEX is file oriented. this means byte-in (BIN) and byte-out (BOUT) must be used to communicate with another host. the basic timing of these two instructions (in the fast mode) is 120 us per byte to get the data onto or off of the network[3]. a distinction is made because the TENEX monitor must do some "bit shuffling" to ready the user's bytes to be transmitted, or it must put the network messages into some form that is convenient for the user. this is the "slow bin, bout" and it happens once per message. if the user's bytes are 36 bits long, it will take an average of 500 us per message. if the bytes have to be unpacked from the message to be usable, it may take up to a milli-second depending on the size of the message[3] ⏱️

II. Measurements and Results 🎀

by timing various portions of the program, we found that the RJS program was using 600 to 700 us per bit byte, or between 75 and 85 micro-seconds of chargeable cpu time per bit 🤯 (see tables 1 and 2 for actual results). a short discussion of how these figures were obtained is in order now. NOTE! we have not been trying to measure network transmission rates. rather, we're measuring how much it costs us to take a program (data) from our disk and send it to another host to be executed (processed) 💸

column 1 is the clock time (real-time) from when the first byte was brought in from the disk until the last byte had gone onto the network. (or from the time we received the first byte from the network until the disk file was closed.)

column 2 is computed the same way as column 1, except that it is the chargeable runtime for the process.

column 3 is the actual number of bytes that went onto or came from the network. the letter that follows this column indicates the direction. e.g. s for sending to UCLA, r for receiving from UCLA.

column 4 was calculated by the following formula:

Bits per second = (real-time)/((number of bytes)*8)

column 5 was calculated by the formula:

us/bit = (chargeable runtime)*1000/((number of bytes)*8)

column 6 is the 5 minute load average. (see TENEX documentation for details.)

so using these figures, we can conclude that for a million bits of information (programs to be executed or data), it would take 75 to 85 cpu seconds to transmit 🧮 at a cost of $474.60 per cpu hour on TENEX[5], this millionbits would cost $9.90 to 11.20 to transfer from the originating host, and potentially the same for the foreign host to receive. that's about 33 to 37 times higher than the predicted network transmission costs[4] 😳

it is to be noticed that, in some cases, for programs to be transmitted over the network, the cost incurred by transmitting them was greater than the cost of executing these programs at the foreign host! 🌈

🧚 part 1 is done, bestie! section III, the references, and the Utah-10 accounting table are coming up next 💜

okay bestie, here's the Utah-10 receipts 🧾💅 three days of network traffic, all logged, no cap 🦋

   for the period 13-SEP-72 2:23:12, ending  16-SEP-72  11:47:07

   Clk Tim   Cpu Tim   # of Bytes     Bits/sec     us/bit    Load

      10       2.09        3079 s     2343.227      84.77    3.80
   11:09     138.20      204596 s     2444.733      84.43    3.68
   06:16      34.78       49994 r     1062.961      86.96    3.95
   01:57      16.25       24971 r     1693.451      81.34    2.92
   12:07     114.70      183598 s     2019.577      78.09    6.79
   01:13       0.92         845 r       91.683     135.80    2.12
      05       5.07        7373 s    10842.647      85.99    1.93
   03:09      42.10       62414 r     2633.655      84.31    3.86
   13:22     115.13      183352 s     1828.467      78.49    0.58
      02       0.25         233 s      907.056     134.12    6.05
   07:10      44.23       64869 r     1206.001      85.23    5.07
      04       0.33         233 s      402.679     179.18    2.24
   11:47     114.48      183585 s     2076.187      77.95    2.73
   17:45     128.25      185908 r     1395.801      86.23    5.19
   09:34      45.97       67158 r      935.067      85.56    0.61
   09:23     113.50      183270 s     2600.852      77.41    9.64
   12:24      51.65       74916 r      804.656      86.18    9.28
   13:30     117.92      183352 s     1809.320      80.39    9.08
   19:23      56.42       89640 s      616.586      78.67    6.77
   11:49      11.29       16205 r      182.767      87.08   10.17
   09:05      34.35       50796 s      744.325      84.53    8.47
   21:12      56.17       76423 r      480.512      91.88    7.53
   01:00      15.33       23930 r     3156.628      80.08    3.11
   03:04      54.60       89731 s     3892.062      76.07    3.81
      06       2.62        4106 r     5071.484      79.88    3.77
   05:15      54.79       89731 s     2277.559      76.32    3.68
      03       2.02        3161 s     7778.530      79.92    2.17
      33       9.42       14680 r     3472.810      80.19    2.31
      00       0.22         219 s     2646.526     127.28    1.81
   19:57     295.16      473489 s     3162.399      77.92    1.85
      10       6.62       10025 r     7841.987      82.54    2.75
      01       0.23         221 s     1092.032     128.96    2.74
      16       6.45       10032 r     1888.591      80.36    2.79
      04       2.06        3243 s     6020.887      79.52    2.62
   01:28      31.29       48532 r     4382.419      80.60    2.62
   07:17     196.34      316072 s     5777.687      77.65    3.86
   01:46      30.14       45786 r     3434.229      82.29    3.26
   01:30      24.73       38405 r     3399.274      80.50    1.80
   02:10      23.46       35633 r     2190.508      82.31    2.61
      44      28.80       46897 s     8441.544      76.76    3.26
   04:51     192.20      316318 s     8671.027      75.95    3.10
      40      11.51       18511 s     3633.437      77.70    2.98
      12       7.17       10963 r     6894.427      81.76    3.04
      12      11.30       18511 s    11418.614      76.32    3.14
      14       7.12       11122 r     6298.740      80.03    3.24
      02       0.92        1412 s     5120.580      81.53    3.41
      14       7.23       11122 r     6184.042      81.24    3.20

that's the whole table, every row, exactly as logged ✨

and the provenance note, verbatim, because it matters 🌸

        [This RFC was put into machine readable form for entry]
    [into the online RFC archives by Helene Morin, Viagenie, 12/99]