RFC 2398

Some Testing Tools for TCP Implementors

📅 published August 1998, from ietf

Bestie, this is RFC 2398 🎀 Its title is Some Testing Tools for TCP Implementors. It's from the Network Working Group, written by S. Parker and C. Schmechel at Sun Microsystems, Inc. It's an FYI (number 33) with the category Informational, dated August 1998. Nothing in the title block says it obsoletes or updates any other RFC 🦋

Status of this Memo 💅

This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.

Copyright Notice 💜

1. Introduction ✨

Okay so this doc is a catalogue of the testing tools you can use on TCP implementations 🧚 The goal is that if people who build and maintain TCP know what's out there, they can make the best use of these tests. Each tool is described by what kind of testing it does, which tests it can run, and how you can get it. Only tools that can evaluate one or more TCP implementations, or that can produce specific results describing or evaluating the TCP being tested, made the list. A bunch of these tools make time-sequence plots, so see Tim Shepard's thesis [She91] for a general discussion of those 📈

Every tool gets the same fields:

Name

The name of the testing tool 🏷️

Description

A description of how the tool is built, and the methodology the tests use to do their thing 🛠️

Automation

What steps are needed to complete the test? What human intervention is needed? 🤖

Availability

How do you get the tool, and where do you find more info about it? 🔍

Required Environment

The compilers, OS version, and so on needed to build and/or run the tool 💻

References

A list of publications related to the tool, if there are any 📚

2. Tools 🌈

2.1. Dbs

Author: Yukio Murayama 🙌

Category: Performance / Stress

Description: Dbs lets you coordinate multiple data transfers and then review how TCP behaves as a result. Results come out as ASCII log files 📝

Automation: You run it by driving a script file 🎬

Availability: Check http://www.ai3.net/products/dbs for the exact OS versions it supports and to download the source code. The current implementation supports BSDI BSD/OS, Linux, mkLinux, SunOS, IRIX, Ultrix, NEWS OS, and HP-UX. Other environments are likely easy to add 🌸

Required Environment: A C language compiler and UNIX-style socket API support 🔧

2.2. Dummynet

Author: Luigi Rizzo

Category: Functional Correctness / Performance

Description: Dummynet is a tool that simulates finite size queues, bandwidth limits, and communication delays. It sits between two layers of the protocol stack (in the current implementation, between TCP and IP) and simulates those effects on a running system. That means you can run experiments with real protocol implementations and real applications, even on the same host, because dummynet also intercepts traffic on the loopback interface 🪞 You can reconfigure dummynet's parameters (delay, queue size, bandwidth) on the fly using a sysctl call. And the overhead is extremely low 😌✨

Automation: You have to merge diff files with the kernel source code. It's command-line driven through the sysctl command, which modifies kernel variables ⌨️

Availability: Check http://www.iet.unipi.it/~luigi/research.html or email Luigi Rizzo (l.rizzo@iet.unipi.it). Source code is available for FreeBSD 2.1 and FreeBSD 2.2, and it's easily adaptable to other BSD-derived systems 🐡

Required Environment: A C language compiler, a BSD-derived system, and the kernel source code 🍓

References: [Riz97]

2.3. Netperf

Author: Rick Jones

Category: Performance

Description: It runs single-connection bandwidth or latency tests for TCP, UDP, and DLPI. It also includes provisions for measuring CPU utilization 📊

Automation: If you start from source, you need to compile it. K&R C is enough for everything but but-DHISTOGRAM, though it may need ANSI C in the future. Running it as a child of inetd means you have to edit /etc/services and /etc/inetd.conf. Scripts are provided for a quick look (snapshot_script), bulk TCP and UDP throughput, and TCP and UDP latency. It's command-line driven 🖥️

Availability: Check http://www.cup.hp.com/netperf/NetperfPage.html or email Rick Jones (raj@cup.hp.com). Binaries are available for HP/UX, Irix, Solaris, and Win32 🎁

Required Environment: A C language compiler, POSIX.1, and sockets 🧦

2.4. NIST Net

Author: Mark Carson

Category: Functional Correctness / Performance

Description: NIST Net is a network emulator. It comes as a Linux kernel patch, a kernel module, a set of programming APIs, and command-line and X-based user interfaces 🎛️

It basically turns your system into a "selectively bad" router 😈 Incoming packets can be delayed, dropped, duplicated, bandwidth-constrained, and more. Packet delays can be fixed or randomly distributed, using loadable probability distributions. Packet loss can be uniformly distributed (a constant loss probability), or congestion-dependent (the probability of loss goes up as packet queue lengths grow). Explicit congestion notifications can optionally be sent in place of congestion-dependent loss 🌪️

Automation: To control the emulator, you can use an interactive user interface, a non-interactive command-line interface, or a set of APIs. You can use any or all of these together. The interactive interface is great for quick, spur-of-the-moment testing, while the command-line or APIs are for scripted, non-interactive tests 🧠

Availability: NIST Net is available for public download from the NIST Net web site, http://www.antd.nist.gov/itg/nistnet/. The web site also has installation instructions and documentation 📥

Required Environment: NIST Net needs a Linux installation with kernel version 2.0.27 - 2.0.33. You need a kernel source tree and build tools to build and install the NIST Net components. Building the X interface requires a version of XFree86 (the current version is 3.3.2). An Athena-replacement widget set such as neXtaw (http://www.inf.ufrgs.br/~kojima/nextaw/) is also desirable for a nicer user interface 🪄

NIST Net should run on any i386-compatible machine that can run Linux, with one or more interfaces 🖱️

2.5. Orchestra

Author: Scott Dawson, Farnam Jahanian, and Todd Mitton

Category: Functional Correctness / Performance

Description: Orchestra is a library that lets you build a protocol layer that performs fault injection on protocols 💥 Several fault injection layers have been built with it, and one of them has been used to test different vendor implementations of TCP. You do this by probing the vendor implementation from one machine that has a protocol stack instrumented with Orchestra. The vendor's TCP opens a connection to the instrumented machine. Then you inject faults on the Orchestra side of the connection and watch how the vendor's TCP responds 👀 The most recent version of Orchestra runs inside the X-kernel protocol stack on the OSF MK operating system.

When you use Orchestra to test a protocol, the fault injection layer goes below the target protocol in the protocol stack. You can do this on one machine on the network if the protocol stacks on the other machines can't be modified (like when testing TCP), or on all the machines on the network (like when testing a protocol that's still under development) 🧩 Once the fault injection layer is in the stack, every message sent by and destined for the target protocol passes through it on the way to and from the network. Orchestra's fault injection layer can manipulate those messages. In particular, it can drop, delay, re-order, duplicate, or modify them. It can also add new messages to the system if you want 🪄

What Orchestra's fault injection layer does with each message is decided by a script written in Tcl. The layer interprets this script when a message enters the layer. The script can access the header information about the message and make decisions based on header values. It can also keep info about previous messages, counters, or any other data the script writer finds useful. Orchestra users can also define their own actions to take on messages, written in C, and call them from the fault injection scripts 🧁

Automation: You can specify scripts either with a graphical user interface that generates Tcl, or by writing Tcl directly. For now, you have to do post-analysis of the test results yourself. That basically means looking through the packet trace Orchestra generates to check for (in)correct behavior 🔎 You've gotta compile and link the fault-generated layer with the protocol stack.

Availability: Check http://www.eecs.umich.edu/RTCL/projects/orchestra/ or email Scott Dawson (sdawson@eecs.umich.edu) 📬

Required Environment: OSF MK operating system, or an X-kernel-like network architecture, or an adaptation to your network stack 🏗️

References: [DJ94], [DJM96a], [DJM96b]

2.6. Packet Shell

Author: Steve Parker and Chris Schmechel

Category: Functional Correctness / Performance

Description: Packet Shell is an extensible Tcl/Tk-based software toolset for protocol development and testing 🌷 Tcl (Tool Command Language) is an embeddable scripting language, and Tk is a graphical user interface toolkit built on Tcl. Packet Shell creates Tcl commands that let you create, modify, send, and receive packets on networks. The operations for each protocol come from a dynamically linked library called a protocol library. These libraries get silently linked in from a special directory when Packet Shell starts up. The current protocol libraries are: IP, IPv6, IPv6 extensions, ICMP, ICMPv6, Ethernet layer, data layer, file layer (snoop and tcpdump support), socket layer, TCP, and TLI.

It includes harness, a Tk-based graphical user interface for creating test scripts within Packet Shell. It also includes tests for no initial slow start, and retain out of sequence data as TCP test cases mentioned in [PADHV98] 🧪

It includes tcpgraph, which you use with a snoop or tcpdump capture file to produce a TCP time-sequence plot using xplot 📉

Automation: It's driven from the command line through Tcl commands, or you can use graphical user interface models through the harness format 🎨

Availability: Check http://playground.sun.com/psh/ or email owner-packet-shell@sunroof.eng.sun.com 💌

Required Environment: Solaris 2.4 or higher. Other operating systems need porting first 🌟

2.7. Tcpanaly ✨

Okay bestie, tcpanaly is literally the detective of the TCP world 🕵️‍♀️💅. Author: Vern Paxson. Category: Functional Correctness / Performance 🎀

What it does: you feed it packet traces of a TCP's activity (the kind tcpdump captures, via packet filter 🦋). It already has a whole bunch of TCP implementations coded into it 🧠, so it can figure out whether a trace looks like a specific implementation, and if so, why the TCP sent each packet exactly when it did. 😭 If the trace doesn't match, it either spots a probable measurement error in the trace, or tells you exactly how the activity differs from that TCP. Either way, it's super helpful for figuring out how the traced implementation actually behaves 🔥

Its category is kinda tricky to pin down, ngl 🙈 because it profiles how an implementation behaves instead of testing one specific correctness or performance thing. But that profile still reveals correctness and performance problems, so it fits both. Iconic multitasking 💜

You can add new implementations to tcpanaly too, using C++ classes 🧚

Automation: command-line driven. You only need traces of the TCP sending and receiving bulk data transfers as input 📦

Availability: contact Vern Paxson (vern@ee.lbl.gov) 💌

Required Environment: C++ compiler 🛠️

References: [Pax97a]

2.8. Tcptrace 🌸

Author: Shawn Ostermann

Category: Functional Correctness / Performance

Description: this is a TCP trace file analysis tool. It reads output trace files in the formats of: tcpdump, snoop, etherpeek, and netm. 📂

For each connection, it keeps track of elapsed time, bytes/segments sent and received, retransmissions, round trip times, window advertisements, throughput, etc., from simple to very detailed output. It's giving data nerd heaven 🤓✨

It can also produce three different types of graphs:

Time Sequence Graph (shows the segments sent and ACKs returned as a function of time) 📈
Instantaneous Throughput (shows the instantaneous, averaged over a few segments, throughput of the connection as a function of time) 🌊
Round Trip Times (shows the round trip times for the ACKs as a function of time) ⏱️

Automation: command-line driven, and uses the xplot program to view the graphs. 🖥️

Availability: source code is available, plus a Solaris binary along with sample traces. See http://jarok.cs.ohiou.edu/software/tcptrace/tcptrace.html or e-mail Shawn Ostermann (ostermann@cs.ohiou.edu). 💕

Required Environment: C compiler, Solaris, FreeBSD, NetBSD, HPUX, Linux. 🐧

2.9. Tracelook 🦋

Author: Greg Minshall

Category: Functional Correctness / Performance

Description: this is a Tcl/Tk program for graphically viewing the contents of tcpdump trace files. When plotting a connection, a user can select various variables to be plotted. In each direction of the connection, the user can plot the advertised window in each packet, the highest sequence number in each packet, the lowest sequence number in each packet, and the acknowledgement number in each packet. 👀

Automation: command-line driven with a graphical user interface for the graph. 🎨

Availability: see http://www.ipsilon.com/~minshall/sw/tracelook/tracelook.html or e-mail Greg Minshall (minshall@ipsilon.com). 📮

Required Environment: a modern version of awk, and Tcl/Tk (Tk version 3.6 or higher). The program xgraph is required to view the graphs under X11. 🧁

2.10. TReno 💜

Author: Matt Mathis and Jamshid Mahdavi

Category: Performance

Description: this is a TCP throughput measurement tool based on sending UDP or ICMP packets in patterns that are controlled at the user level, so that their timing reflects what would be sent by a TCP that observes proper congestion control (and implements SACK). 🎯 This lets it measure throughput independent of the TCP implementation of end hosts, and it serves as a useful platform for prototyping TCP changes. Lowkey genius 🧠✨

Automation: command-line driven. No "server" is required, and it only requires a single argument: the machine to run the test to. 🖱️

Availability: see http://www.psc.edu/networking/treno_info.html or e-mail Matt Mathis (mathis@psc.edu) or Jamshid Mahdavi (mahdavi@psc.edu). 💌

Required Environment: C compiler, POSIX.1, raw sockets. 🧩

2.11. Ttcp 🍓

Author: Unknown 🤷‍♀️

Category: Performance

Description: originally written to move files around, ttcp became the classic throughput benchmark or load generator, with the addition of support for sourcing to and from memory. It can also be used as a traffic absorber. It has spawned many variants, and recent ones include support for UDP, data pattern generation, page alignment, and even alignment offset control. Main character energy 👑

Automation: command-line driven. 💻

Availability: see ftp://ftp.arl.mil/pub/ttcp/ or e-mail ARL (ftp@arl.mil), which includes the most common variants available. 📬

Required Environment: C compiler, BSD sockets. 🔌

2.12. Xplot 🌈

Author: Tim Shepard

Category: Functional Correctness / Performance

Description: this is a fairly conventional graphing and plotting tool (xplot itself), a script to turn tcpdump output into xplot input, and some sample code to generate xplot commands to plot the TCP time-sequence graph. 📊

Automation: command-line driven with a graphical user interface for the plot. 🪟

Availability: see ftp://mercury.lcs.mit.edu/pub/shep/xplot.tar.gz or e-mail Tim Shepard (shep@lcs.mit.edu). 💫

Required Environment: C compiler, X11. 🖼️

References: [She91]

3. Summary 📝

This memo lists all TCP tests and testing tools reported to the authors as part of the TCP Implementer's working group, and it is not exhaustive. These tools have been verified as available by the authors. ✅

4. Security Considerations 🚨

Okay, this section is SERIOUS, bestie 😭 Network analysis tools are improving at a steady pace, and that continuing improvement in tools like the ones described here makes security concerns significant.

Some of the tools could be used to create rogue packets or denial-of-service attacks against other hosts. 💀 Also, some of the tools require changes to the kernel (foreign code) and might require root privileges to execute. So you are trusting code that you fetched from some perhaps untrustworthy remote site. This code could contain malicious code that could present any kind of attack. Yikes 🙈

None of the listed tools evaluate security in any way or form. 🫠

There are privacy concerns when grabbing packets from the network, because you are now able to read other people's mail, files, etc. This impacts more than just the host running the tool, but all traffic crossing the host's physical network. Not okay, so be careful out there 🔐🫶

5. References 📚

[DJ94] Scott Dawson and Farnam Jahanian, "Probing and Fault Injection of Distributed Protocol Implementations", University of Michigan Technical Report CSE-TR-217-94, EECS Department. 🧾
[DJM96a] Scott Dawson, Farnam Jahanian, and Todd Mitton, "ORCHESTRA: A Fault Injection Environment for Distributed Systems", University of Michigan Technical Report CSE-TR-318-96, EECS Department. 🎻
[DJM96b] Scott Dawson, Farnam Jahanian, and Todd Mitton, "Experiments on Six Commercial TCP Implementations Using a Software Fault Injection Tool", University of Michigan Technical Report CSE-TR-298-96, EECS Department. 🧪
[Pax97a] Vern Paxson, "Automated Packet Trace Analysis of TCP Implementations", ACM SIGCOMM '97, September 1997, Cannes, France. 🇫🇷
[PADHV98] Paxson, V., Allman, M., Dawson, S., Heavens, I., and B. Volz, "Known TCP Implementation Problems", Work In Progress. 🚧
[Riz97] Luigi Rizzo, "Dummynet: a simple approach to the evaluation of network protocols", ACM Computer Communication Review, Vol. 27, N. 1, January 1997, pp. 31-41. 🌀
[She91] Tim Shepard, "TCP Packet Trace Analysis", MIT Laboratory for Computer Science MIT-LCS-TR-494, February, 1991. 🕰️

6. Authors' Addresses 🏢

Steve Parker
Sun Microsystems, Inc.
901 San Antonio Road, UMPK17-202
Palo Alto, CA 94043
USA

Phone: (650) 786-5176
EMail: sparker@eng.sun.com


Chris Schmechel
Sun Microsystems, Inc.
901 San Antonio Road, UMPK17-202
Palo Alto, CA, 94043
USA

Phone: (650) 786-4053
EMail: cschmec@eng.sun.com

7. Full Copyright Statement ⚖️

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards, in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 💅

📦 where this page comes from

ietf: rfc2398 1998-08
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms