RFC 1025

TCP and IP bake off

📅 published September 1987, from ietf

TCP AND IP BAKE OFF 🎀

  • Network Working Group 💜
  • Request for Comments: 1025 📄
  • Author: J. Postel, ISI 🧑‍💻
  • Date: September 1987 📅
  • Category, obsoletes, updates: the header doesn't list a category, and it doesn't say this obsoletes or updates anything 🙈

Status of This Memo ✨

Okay so this memo walks through the procedures, scoring, and tests from the TCP and IP bake offs that happened during the early development of these protocols 💅 These procedures and tests might still be super useful when you're testing brand-new TCP and IP modules 🧪 And distribution is unlimited, so share it with everyone 🫶

Introduction 🦋

Back in the early days of TCP and IP, there were barely any implementations and the specs were still changing 🌱 The only way to tell if your implementation was "correct" was to test it against other implementations and then argue that your results proved you did the right thing 😤 Honestly, those tests and debates could change the spec just as easily as they changed the implementation 🤯

Sometimes everyone got together, gathered all the known implementations, and ran through a set of tests to show off the N squared connectivity and the correct handling of the tricky cases 🍓 Those events were called "Bake Offs" 🥧

An early version of the test list showed up in IEN-69 (October 1978) 📜 Four TCP implementations were demoed at the Defense Communication Engineering Center in Reston, Virginia on 4 December 1978, and that was reported in IEN-70 (December 1978) 🌈 Then six implementations went head to head on 27-28 January 1979 at USC-Information Sciences Institute in Marina del Rey, California, reported in IEN-77 (February 1979) 🌴 And in April 1980 they ran a distributed bake off over the network itself, reported in IEN-145 (May 1980) 🛰️

The next section reproduces (with very slight editing) the procedure, tests, and scoring of the April 1980 Bake Off 🧚

Procedure 💅

Here's the procedure for the TCP and IP Bake Off! Every implementor of a TCP and IP needs to run the following tests and report the results ✨ Usually that means using a test program or a user Telnet program to open connections to your own TCP implementation or to other ones 💖

Some tests get extra spicy with a "flakeway," which is a purposely flakey gateway 🙃 It should have control parameters you can adjust while it's running, so you can set a percentage of datagrams to drop, a percentage to corrupt and still pass on, and a percentage to reorder so they arrive in a different order than they were sent 🌪️

A lot of these points apply for each distinct TCP you contact. (For example, in the Middleweight Division there's a possibility of 20 points for each other TCP in the Bake Off 👑)

Note Bene: Checksums must be enforced. No points will be awarded if the checksum test is disabled. 🙅‍♀️

Featherweight Division 🪶

  • 1 point for talking to yourself (opening a connection).
  • 1 point for saying something to yourself (sending and receiving data).
  • 1 point for gracefully ending the conversation (closing the connection without crashing).
  • 2 points for repeating the above without reinitializing the TCP.
  • 5 points for a complete conversation via the testing gateway.

Middleweight Division 🥊

  • 2 points for talking to someone else (opening a connection).
  • 2 points for saying something to someone else (sending and receiving data).
  • 2 points for gracefully ending the conversation (closing the connection without crashing).
  • 4 points for repeating the above without reinitializing the TCP.
  • 10 points for a complete conversation via the testing gateway.

Heavyweight Division 🔥

  • 10 points for being able to talk to more than one other TCP at the same time (multiple connections open and active simultaneously with different TCPs).
  • 10 points for correctly handling urgent data.
  • 10 points for correctly handling sequence number wraparound.
  • 10 points for correctly being able to process a "Kamikaze" packet (AKA nastygram, christmas tree packet, lamp test segment, et al.). That is, correctly handle a segment with the maximum combination of features at once (e.g., a SYN URG PUSH FIN segment with options and data). 💀
  • 30 points for KOing your opponent with legal blows. (That is, operate a connection until one TCP or the other crashes; the surviving TCP has KOed the other. Legal blows are segments that meet the requirements of the specification.) 👊
  • 20 points for KOing your opponent with dirty blows. (Dirty blows are segments that violate the requirements of the specification.) 😈
  • 10 points for showing your opponent's checksum test is faulty or disabled. 🕵️‍♀️

Host & Gateway IP Division 🌐

  • 25 points for doing fragmentation and reassembly.
  • 15 points for doing loose source route option.
  • 15 points for doing strict source route option.
  • 10 points for doing return route option.
  • 10 points for using source quench messages.
  • 10 points for using routing advice messages.
  • 5 points for doing something with the type of service.
  • 5 points for doing something with the security option.
  • 5 points for doing something with the timestamp option.
  • 5 points for showing that a gateway forwards datagrams without decreasing the time to live (showing a gateway is faulty).
  • 5 points for showing that a gateway forwards datagrams with the time to live equal zero (showing a gateway is faulty).
  • 10 points for showing that a gateway or hosts checksum test is faulty or disabled (showing a gateway is faulty).

Bonus Points 🏆

  • 10 points for the best excuse.
  • 20 points for the fewest excuses.
  • 30 points for the longest conversation.
  • 40 points for the most simultaneous connections.
  • 50 points for the most simultaneous connections with distinct TCPs.

Tests 🧪

The following tests have been identified for checking the capabilities of a TCP implementation. They might be useful when you're trying to KO an opponent 💀

  1. Single connection. Open & close a single connection many times.
  2. Multi connections. Open several connections simultaneously. Two connections to the same socket (i.e., a-b and a-c) check proper separation of data.
  3. Half Open Connection. Open a connection, crash local TCP and attempt to open same connection again.
  4. Piggy-back Loop. Open connections via Telnet.
         user telnet--->TCP--->IP--->net--->IP--->TCP--->server telnet
                                                               |
                                                               V
         server telnet<---TCP<---IP<---net<---IP<---TCP<---user telnet
              |
              V
         user telnet--->...
  1. Maximum connections. Open connections between a pair of TCP until refused or worse.
  2. Refused connection. Open a connection to a non-accepting socket, does it get refused?
  3. Zero Window. Try to send data to a TCP that is presenting a zero window.
  4. Fire Hose. Make many connections to data source ports, or connections to a data sink and send as fast as you can.
  5. Urgent Test. Try to send data to a user program that only receives data when in urgent mode.
  6. Kamikazi Segment. Send and receive nastygrams. A nastygram is a segment with SYN, EOL, URG, and FIN on and carrying one octet of data. 💥
  7. Sequence Wraparound. Test proper functioning when sequence numbers (a) pass 2**31 (i.e., go from plus to "minus") and (b) pass 2**32 (i.e., go from 2**32-1 to 0). 🔄
  8. Buffer size. With buffer size not equal to one, send data in segments of various sizes, use urgent occasionally.
  9. Send a nastygram into a half open connection when the sequence number is about to wrap around. 🫣

New Ideas 🌸

The above tests check basic operation and some of the tricky cases 🧐 But they don't look at performance at all, or check whether some of the more recently developed ideas have been implemented 👀

New Mechanisms ⚙️

  1. The John Nagel Procedures (RFC-896).
  2. The Van Jacobson Procedures (slow start, RTT measurements, etc).
  3. The SQuID Procedures (RFC-1016).

Performance Tests 📊

Performance tests are tricky to specify, because the results depend so much on the state of the environment of the test 🌦️ Here are a few possibilities:

  1. FTP Throughput: Send a 1 megabyte file to a locally nearby machine on an Ethernet, measuring the elapsed time.
  2. FTP Throughput: Send a 1 megabyte file to a locally nearby machine on an ARPANET, measuring the elapsed time.
  3. NETBLT Throughput: Send a 1 megabyte file to a locally nearby machine on an Ethernet, measuring the elapsed time.
  4. NETBLT Throughput: Send a 1 megabyte file to a locally nearby machine on an ARPANET, measuring the elapsed time.
  5. Character Test: Use a test program to send a character via TCP to the Echo Server (RFC-862), and time the round trip (from the time the character is sent until the echo is returned to the test program). ⏱️

Appendix 🎀

For History Buffs Only: 📚

The following item was in the original 1980 tests, but it's been moved to this appendix since it no longer applies 🕰️

  • 10 points for correctly handling rubber baby buffer bumpers in both directions (End of Letter sequence number adjustments). 🍼
📦 where this page comes from