RFC 1106

TCP big window and NAK options

๐Ÿ“… published June 1989, from ietf

RFC 1106: TCP Big Window and Nak Options ๐ŸŽ€

Bestie, this one is a vibe ๐Ÿ’… It's from June 1989 and it's all about how TCP can handle really big, really long-distance connections without wasting bandwidth. Here's the title block, all in one place:

  • RFC number: 1106
  • Title: TCP Big Window and Nak Options
  • Author: R. Fox, Tandem
  • Working group: Network Working Group
  • Date: June 1989
  • Category: Experimental
  • Obsoletes / Updates: none listed in this header

Status of This Memo โœจ

Okay so this memo is about two extensions to TCP that make things run more efficiently over a network with a high bandwidth*delay product ๐Ÿš€ The extensions were actually implemented and tested using resources at NASA, so they're not just theory.

This is an Experimental Protocol ๐Ÿงช It is not being proposed as an Internet standard. It's more of a starting point for further research, which I'm honestly obsessed with. Distribution of this memo is unlimited ๐ŸŒˆ

Abstract ๐Ÿฆ‹

Two extensions to TCP are described in this RFC so that it works more efficiently over networks with a high bandwidth*delay product. The big thing that still needs solving is congestion versus noise ๐Ÿ˜ญ This memo touches on that issue, but more research is still needed to see whether these extensions work across the Internet as a whole and not just on high bandwidth*delay networks.

The document does show how to use these options in an isolated satellite network. The goal is to use that special medium more efficiently and take bulk data transfers off the links that people need for interactive use. Main character energy for satellites ๐Ÿ›ฐ๏ธโœจ

1. Introduction ๐ŸŒธ

Recent work on TCP has given great performance gains over a bunch of different network paths [1]. Still, these changes don't work well on paths with a large round trip delay (like a satellite with a 600 ms round trip delay) or with a very large bandwidth (like a transcontinental DS3 line) ๐Ÿ˜ฉ Both of these networks have a higher bandwidth*delay product, over 10**6 bits, while TCP is currently limited to 10**5 bits.

So what's bandwidth*delay anyway? It's the amount of data that can be unacknowledged at once so that all of the network's bandwidth is being used by TCP. People also call this "filling the pipe" [2] ๐Ÿชฃ When the pipe is full, the sender can always put data onto the network, the receiver always has something to read, and neither end has to wait around for the other one. It's giving efficiency ๐Ÿ’–

After the last batch of algorithm improvements to TCP, performance over high bandwidth*delay networks is still really bad ๐Ÿ’€ It looks like no algorithm changes alone will make a meaningful difference on these networks. They need an extension to the protocol itself. This RFC discusses two possible TCP options for that purpose ๐ŸŽ€

The two options implemented and discussed in this RFC are:

  1. NAKs ๐Ÿ™…โ€โ™€๏ธ This extension lets the receiver tell the sender that a packet of data didn't arrive and needs to be resent. It's useful on any network path, whether it has a high or low bandwidth*delay product, as long as the path has periodic errors like lost packets, noisy links, or packets dropped because of congestion. The information from this option is advisory. If the sender ignores it, it has no effect on TCP whatsoever ๐Ÿซ 
  2. Big Windows ๐ŸชŸ This option expands the current 16 bit (64 Kbytes) TCP window to 32 bits. Of those 32 bits, 30 bits (over 1 gigabytes) are allowed for the receive window. (The maximum window size in TCP exists because TCP has to be able to tell old data from new data. For a good explanation, see [2].) No changes are needed to the standard TCP header [6] ๐Ÿ™Œ The 16 bit field in the TCP header that carries the receive window stays exactly the same. The 32 bit receive window comes from an option that holds the upper half of the window. This option is what's needed to fill big data pipes, like a satellite link ๐Ÿ›ฐ๏ธ

Here's the roadmap, bestie ๐Ÿ—บ๏ธ Section 2 goes into more detail on the NAK option, and Section 3 goes into more detail on the big window option. Section 4 covers the other effects of using the big windows and NAK features together. It includes a short discussion of how congestion versus noise affects TCP, plus some options for satellite networks. Section 5 is the conclusion, with some hints about what future development might happen at NASA. After that there's an appendix with some test results ๐Ÿ“Š

2. NAK Option ๐Ÿ™…โ€โ™€๏ธ

Any packet loss on a high bandwidth*delay network hits throughput really hard, and it's all because of TCP's simple acknowledgement scheme ๐Ÿ˜ญ TCP always acks the stream of data that has been received successfully, and it tells the sender the next byte it expects in the stream. If a packet gets lost but the packets after it arrive, the current protocol has no way to tell the sender "hey, I missed one, but I got the ones after it."

So TCP resends all of the data over again, either after a timeout or after the sender suspects a loss because of the duplicate ack algorithm [1]. It keeps doing this until the receiver gets the lost packet and can ack it along with the packets that came after it. On a normal low bandwidth*delay network, this isn't a big deal as long as the timeout period is set short enough ๐Ÿ‘

On a long delay network, like a T1 satellite channel, it's a disaster ๐Ÿ’€ By the time the lost packet gets resent and the ack comes back, the TCP window has been used up. The sender and receiver both end up stalled, waiting for the packet and the ack to fully travel through the data pipe. The pipe empties out, and the sender has to refill it after the ack comes back.

This causes a minimum of 3*X bandwidth loss, where X is the one way delay of the medium. It can be a lot higher depending on how big the timeout period is and what the bandwidth*delay product is. The 3*X breaks down like this: 1X for the packet to be resent, 1X for the ack to be received, and 1X for the next packet to reach the destination โœจ This math assumes the window size is much smaller than the pipe size (window = 1/2 data pipe or 1X), which is the normal case with today's TCP window limit on long delay networks like a T1 satellite link.

An attempt to reduce that wasted 3*X was introduced in [1]. The sender resends a packet after it notices a number of consecutively received acks that completely acknowledge data that was already acknowledged. On a typical network, this brings the lost bandwidth down to almost nothing ๐ŸŽ‰ The packet gets resent before the TCP window runs out, and since the data pipe is much smaller than the TCP window, the pipe never empties and no bandwidth is lost.

On a high delay network, though, the reduction is minimal, and the lost bandwidth is still significant ๐Ÿ˜ฉ On a very noisy satellite, the lost bandwidth is very high (see the appendix for some performance figures), and performance is very poor.

There are two methods for telling the sender about lost data: selective acknowledgements and NAKs ๐Ÿซถ Selective acknowledgements have been researched in a number of experimental protocols, including VMTP [3], NETBLT [4], and SatFTP [5]. The idea behind selective acks is that the receiver tells the sender which pieces it received, so the sender can resend the data that hasn't been acked but was already sent once. NAKs, on the other hand, tell the sender that a particular packet of data needs to be resent.

Selective acks have a few disadvantages, bestie ๐Ÿ™ˆ In some of the protocols above, the receiver waits a certain amount of time before sending the selective ack so that acks can be bundled together. That delay can waste some bandwidth, and it needs more complex state information than a simple nak. Even if the receiver doesn't bundle the selective acks and sends them as soon as it notices lost packets, more complex state information is still needed to figure out which packets have been acked and which ones need to be resent. With naks, only the immediate data needed to move the left edge of the window is naked, which almost completely gets rid of state information ๐Ÿง โœจ

Selective acks do have one advantage over naks. If the link is very noisy and packets are getting lost close together, the sender finds out about all of the missing data at once. It can send all of that missing data right away to move the left window edge in the acknowledge number of the TCP header, which keeps the data pipe flowing ๐ŸŒŠ With naks, the sender hears about lost packets one at a time, so it ends up processing extra packets compared to selective acks.

Here's the plot twist though ๐Ÿ˜Œ Empirical studies have shown that most lost packets occur far enough apart that the advantage of selective acks over naks is rarely seen. Also, if naks are sent as soon as a packet is determined to be lost, the advantage of selective acks becomes no more than possibly a more aesthetic algorithm for handling lost data. It doesn't offer any gains over the naks described in this paper. That's the reason the simplicity of naks was chosen over selective acks for the current implementation. Simple really is iconic ๐Ÿ‘‘

2.1. Implementation details โœจ

okay so bestie, picture this ๐ŸŒธ. When the receiver notices a gap between the sequence number it was expecting and the one that actually showed up, it can assume the data in between is either running late or lost forever ๐Ÿ˜ญ. Here's the tea: it literally can't tell which one is happening, so a nak should be sent in the TCP option field. Naking a packet that was still on its way makes the sender resend it anyway, which wastes one packet's worth of bandwidth ๐Ÿ’ธ. But this is pretty rare, so that wasted bandwidth is tiny compared to the loss you'd get from not sending a nak when the packet was never coming. Lowkey a great trade-off ๐Ÿ’…. The option looks like this:

      +========+=========+=========================+================+
      +option= + length= + sequence number of      + number of      +
      +   A    +    7    +  first byte being naked + segments naked +
      +========+=========+=========================+================+

This option carries the first sequence number not received, plus a count of how many segments of bytes needed to be resent. Here, a segment is the size of the current TCP MSS being used for the connection ๐Ÿงš. A nak is more of an advisory piece of info than a guarantee, so sending one is unreliable, and there's no way to retransmit a nak yet ๐Ÿ™ˆ. That's not a dealbreaker though, it's just how it is for now.

When the sender gets the option, it can either do nothing, or resend the missing data right away and then keep going from where it left off before the nak came in ๐Ÿƒโ€โ™€๏ธ. The receiver keeps track of the last nak it sent so it won't repeat the same one. If it did repeat, the protocol could get stuck in a mode where every single data reception makes the receiver nak the first missing data frame ๐Ÿ˜ฉ. And since the data pipe can be super huge by the time the first nak gets read and answered, a bunch of naks would already be on their way. The sender doesn't know they're repeats, so it resends the data every time, wasting bandwidth on pointless retransmissions of the same piece of data ๐Ÿ’€.

Since the nak is unreliable, one could get damaged and never reach the sender. If that happens, we just let TCP recover through its normal means ๐Ÿซถ. Empirical data has shown that the nak is pretty unlikely to get lost, so this advisory nak option works quite well โœจ.


3. Big Window Option ๐Ÿ‘‘

Okay so TCP has a 16 bit window limit built right into the protocol ๐Ÿ™ƒ. That caps the amount of outstanding unacknowledged data at 64 Kbytes. We already saw that some networks have a pipe bigger than 64 Kbytes. A T1 satellite channel and a cross country DS3 network with a 30ms delay both have data pipes way bigger than 64 Kbytes ๐Ÿ›ฐ๏ธ. So even on a perfectly conditioned link, with no bandwidth wasted to errors, the data pipe won't get filled and bandwidth gets wasted anyway ๐Ÿ˜ค.

What we need is the ability to send more unacknowledged data, and that's done with bigger windows, bigger than the current 16 bit limit ๐Ÿฆ‹. This option expands the window to 30 bits, which is over 1 gigabytes, by literally stretching the window mechanism TCP already uses. The added option carries the upper 15 bits of the window, while the lower 16 bits keep going where they normally go [6] in the TCP header.

A TCP session only uses the big window options if both sides agree to use them. If they don't, the option just isn't used and the normal 16 bit windows get used instead. Once the two sides agree on big windows, every packet after that is expected to carry the window option with the current upper 15 bits of the window ๐Ÿ’–. The negotiation happens during the SYN and SYN ACK segments of the TCP connection startup process. The originator of the connection includes the following option in the SYN segment:

              1 byte    1 byte      4 bytes
              +=========+==========+===============+
              +option=B + length=6 + 30 bit window +
              +=========+==========+===============+

If the other end of the connection wants to use big windows, it includes the same option back in the SYN ACK segment it has to send ๐Ÿค. At this point both sides have agreed to use big windows, and the specified windows get used. Worth noting: the SYN and SYN ACK segments still use the small windows, and only once the big window option has been negotiated do the bigger windows kick in ๐ŸŒˆ.

Once both sides have agreed to use 32 bit windows, the protocol works just as it did before with no difference in operation, even in the event of lost packets ๐Ÿ˜Œ. That's true because the rcv_wnd and snd_wnd variables of TCP hold the 16 bit windows until the big window option is negotiated, and then they get replaced with the appropriate 32 bit values. So the use of big windows becomes part of the state information kept by TCP ๐Ÿง .

Other folks have presented other ways to expand the windows, including a window multiple [2] or streaming [5]. But this solution is honestly more elegant ๐Ÿ‘‘. It's a true extension of the window, which one day could easily become part of the protocol itself, instead of just being an option bolted onto it.

3.1. How does it work

Once a connection has decided to use big windows, every succeeding packet must contain the following option:

        +=========+==========+==========================+
        +option=C + length=4 + upper 15 bits of rcv_wnd +
        +=========+==========+==========================+

With every segment sent, the sender supplies the size of its receive window ๐Ÿ“ฆ. If the connection is only using 16 bits, this option isn't supplied. Otherwise, the lower 16 bits of the receive window go into the TCP header where they currently reside [6], and the upper 15 bits of the window get put into the data portion of option C. When the receiver processes the packet, it must first reform the window and then process the packet as it would in the absence of the option ๐Ÿ”.

3.2. Impact of changes

Implementing the first version of the big window option needed very little change to the source ๐Ÿ› ๏ธ. State information has to be added to the protocol to determine whether the big window option is used, and all the 16 bit variables that dealt with window information now have to become 32 bit quantities. A future document will describe in more detail the changes required to the 4.3 bsd TCP source code. Test results for the window change only are presented in the appendix ๐Ÿ“Š.

When expanding 16 bit quantities to 32 bit quantities in the TCP control block in the source (4.3 bsd source), the structure may become larger than the mbuf used to hold it. Care must be taken to ensure this doesn't happen on your system, or undetermined events may take place ๐Ÿ˜ฑ.


4. Effects of Big Windows and Naks when used together โœจ

With big windows alone, transfer times over a satellite were quite impressive when there were no introduced errors ๐Ÿš€. However, when an error simulator was used to create random errors during transfers, performance went down extremely fast ๐Ÿ“‰. When the nak option was added to the big window option, performance in the face of errors went up some, but not to the level that was expected. This section will discuss some issues that were overcome to produce the results given in the appendix ๐Ÿ’….

4.1. Window Size and Nak benefits

Without errors, the window size required to keep the data pipe full is equal to the round trip delay * throughput desired, which is the data pipe bandwidth (called Z from now on) ๐Ÿงฎ. This and other calculations assume that processing time of the hosts is negligible. In the event of an error (without NAKs), the window size needs to become larger than Z in order to keep the data pipe full while the sender is waiting for the ack of the resent packet. If the window size is equal to Z, and we assume the retransmission timer is equal to Z, then when a packet is lost, the retransmission timer will go off as the last piece of data in the window is sent โฐ. In this case, the lost piece of data can be resent with no delay.

The data pipe will empty out, because it takes 1/2Z worth of data to get the ack back to the sender, plus an additional 1/2Z worth of data to get the data pipe refilled with new data. This causes the required window to be 2Z: 1Z to keep the data pipe full during normal operations, and 1Z to keep the data pipe full while waiting for a lost packet to be resent and acked ๐Ÿซ .

If the same scenario in the last paragraph is used with the addition of NAKs, the required window size still needs to be 2Z to avoid wasting any bandwidth in the event of a dropped packet. This seems to mean the nak option doesn't provide any benefits at all ๐Ÿ™ƒ. Testing showed that the retransmission timer was larger than the data pipe, and in the event of errors it became much bigger than the data pipe, because of the retransmission backoff. Thus, the nak option bounds the required window to 2Z, such that in the event of an error there is no lost bandwidth, even with the retransmission timer fluctuations. The results in the appendix show that by using naks, the bandwidth waste associated with the retransmission timer facility is eliminated ๐ŸŽ‰.

4.2. Congestions vs Noise

okay bestie, this one's a whole mood ๐Ÿฅบ when you're running NAKs and big windows together, you have to figure out if a packet vanished because the network was jammed (congestion) or because the wire was noisy and mangled it (noise). those need totally different responses, and mixing them up is a disaster ๐Ÿ’€

so the recent algorithm enhancements [1] added slow start. whenever a transfer is starting on a connection, or right after a packet gets dropped, the effective send window shrinks to something tiny (typically equal to the MSS being used) ๐ŸŒธ why? so a brand-new connection doesn't instantly slam the network, and so an existing connection backs off when a packet was dropped for congestion, giving the network a chance to clear up โœจ now, if a big-window connection loses a packet to the medium (a packet corrupted by an error), the worst possible move is closing the send window so it can only send 1 packet and has to crawl back up with slow start to a full window's worth of data ๐Ÿ˜ฉ that would quickly kill the usefulness of the big window and nak options over lossy links.

on the flip side, if a packet got dropped because of congestion but the sender assumes it was noise, the sender keeps blasting huge amounts of data ๐Ÿ˜ฌ which keeps the congestion going, more packets get dropped, and that part of the network collapses ๐Ÿ’€ in this case the sender would want to back off from the current window limit. using the current slow start mechanism over a satellite builds the window up too slowly [1]. possibly a better solution would be for the window to be opened 2*Rlog2(W) instead of R*log2(W) [1] (open window by 2 packets instead of 1 for each acked packet). this cuts down on wasted bandwidth by opening the window way quicker while still giving the network a chance to clear up ๐Ÿฆ‹ more experimentation is needed to find the optimal rate for opening the window, especially when large windows are being used.

the current recommendation for TCP is to use slow start on any lost packet. if an application knows it'll be using a satellite with a high error rate, forcing slow start on every dropped packet doesn't make sense ๐Ÿ™„ so the application should be able to choose what happens when a packet is lost. in the BSD environment, a setsockopt call should be provided so the app can tell TCP to handle lost packets in a special way for that particular connection ๐Ÿ’… if the known error rate of a link is small, then slow start with the modified rate from above causes the bandwidth loss to be very small compared to the bandwidth actually used. in that case the setsockopt call should not be used. what's really needed is a way for a host to determine if packets are being dropped due to congestion or noise, so it can do the right thing ๐Ÿง  this will require a mechanism like source quench, and more experimentation is needed to get a solid definition for using it. now, some people believe that using source quench to avoid congestion only adds to the problem instead of helping suppress it ๐Ÿ™ƒ

the TCP used to gather the results in the appendix for the big window with nak experiment assumed that lost packets came from noise and not congestion. that assumption showed how to make the current TCP work in that kind of environment โœจ the actual satellite used in the experiment (when the satellite simulator wasn't used) only had an error rate around 10e-10. with an error rate that low, when big windows are used over the link, TCP should use slow start for all lost packets with the 2*Rlog2(W) rate discussed above. under most situations with long delay networks (transcontinental DS3 networks using fiber with very low error rates, or satellite links with low error rates), big windows and naks should be used with the assumption that lost packets are the result of congestion, until a better algorithm is devised [7] ๐Ÿ’–

one more thing: while testing the effects of slow start over a satellite link, the retransmission timer was sometimes set SO restrictive that milliseconds before a naked packet's ack came back, the timer would go off because of a timed packet inside the send window ๐Ÿ˜ญ

the timer was set at the round trip delay of the network, which left no time for packet processing. if that timer went off due to congestion, then backing off is the right thing to do. otherwise, to avoid the scenario found by experimentation, the transmit timer should be set a little longer so the retransmission timer doesn't go off too early. care must be taken to make sure the right thing happens in the implementation in question, so a packet isn't retransmitted too soon and blamed on congestion when the ack is actually on its way ๐Ÿš—๐Ÿ’จ

4.3. Duplicate Acks

another problem found with the 4.3bsd implementation is in the area of duplicate acks ๐Ÿ‘€ when the sender of data receives a certain number of acks (3 in the current Berkeley release) that acknowledge data that was already acked before, it assumes a packet has been lost. it resends the one packet it assumes is lost, closes its send window as if the network is congested, and uses the slow start algorithm mentioned above to reopen the window. this facility is no longer needed, since the sender can use the reception of a nak as its indicator that a particular packet was dropped ๐Ÿ™Œ if the nak packet is lost, the retransmit timer will go off and the packet gets retransmitted by normal means. if a sender's algorithm keeps counting duplicate acks, it may end up receiving many duplicate acks after it has already resent the packet because of a nak, thanks to the large size of the data pipe ๐Ÿ˜ต by receiving all of those duplicate acks, the sender can find itself doing nothing but resending the same packet of data unnecessarily while keeping the send window closed for absolutely no reason ๐Ÿ™ƒ by removing this feature from the implementation, a user can expect a satellite connection to work much better in the face of errors, and other connections should not see any performance loss, but a slight improvement in performance if anything at all ๐ŸŽ€


5. Conclusion

this paper described two new options that, if used, will make TCP a more efficient protocol in the face of errors, and a more efficient protocol over networks that have a high bandwidth*delay product, without decreasing performance over more common networks ๐ŸŒˆ if a system that implements the options talks with one that doesn't, the two systems should still be able to communicate with no problems. this assumes the system doesn't use the option numbers defined in this paper in some other way, and doesn't panic when faced with an option it doesn't implement. currently at NASA there are many machines that don't implement either option and communicate just fine with the systems that do ๐Ÿš€

the drive for implementing big windows has been the direct result of trying to make TCP more efficient over large delay networks [2,3,4,5] such as a T1 satellite. however, another practical use of large windows is becoming more apparent as local area networks get faster and support much larger MTUs ๐ŸŒŸ Hyperchannel, for instance, has been said to be able to support 1 Mega bit MTUs in its new line of products. with the current implementation of TCP, efficient use of Hyperchannel isn't achieved the way it should be, because the physical medium's MTU is larger than the maximum window of the protocol being used ๐Ÿ˜ค by increasing the TCP window size, better utilization of networks like Hyperchannel will be gained instantly, because the sender can send 64 Kbyte packets (IP limitation) without having to operate in a stop and wait fashion ๐Ÿ’… future work is starting to increase the IP maximum datagram size, so even better utilization of fast local area networks will be seen, with the TCP/IP protocols able to send large packets over mediums with very large MTUs. this will hopefully eliminate the network protocol as the bottleneck in data transfers, while workstations and workstation file system technology advances even more than it already has ๐Ÿงš

an area of concern when using the big window mechanism is the use of machine resources ๐Ÿ˜ฌ when running over a satellite and a packet is dropped such that 2Z (where Z is the round trip delay) worth of data is unacknowledged, both ends of the connection need to be able to buffer the data using machine mbufs (or whatever mechanism the machine uses), which is usually a valuable and scarce commodity. if the window size isn't chosen properly, some machines will crash when the memory is all used up, or it will keep other parts of the system from running. so setting the window to some fairly large arbitrary number is not a good idea, especially on a general purpose machine where many users log on at any time ๐Ÿ™ˆ what's currently being engineered at NASA is the ability for certain programs to use the setsockopt feature in 4.3bsd to ask for big windows, so the average user may not have access to the large windows. that limits the use of big windows to applications that absolutely need them, and protects a valuable system resource ๐Ÿ”’


6. References

[1] Jacobson, V., "Congestion Avoidance and Control", SIGCOMM 88, Stanford, Ca., August 1988.

[2] Jacobson, V., and R. Braden, "TCP Extensions for Long-Delay Paths", LBL, USC/Information Sciences Institute, RFC 1072, October 1988.

[3] Cheriton, D., "VMTP: Versatile Message Transaction Protocol", RFC 1045, Stanford University, February 1988.

[4] Clark, D., M. Lambert, and L. Zhang, "NETBLT: A Bulk Data Transfer Protocol", RFC 998, MIT, March 1987.

[5] Fox, R., "Draft of Proposed Solution for High Delay Circuit File Transfer", GE/NAS Internal Document, March 1988.

[6] Postel, J., "Transmission Control Protocol - DARPA Internet Program Protocol Specification", RFC 793, DARPA, September 1981.

[7] Leiner, B., "Critical Issues in High Bandwidth Networking", RFC 1077, DARPA, November 1989.

7. Appendix โœจ

Okay so bestie, both options got actually built and tested ๐ŸŽ€ Here are the receipts ๐Ÿ’… This section has some performance numbers that back up using these two options. The satellite channel was a 1.544 Mbit link with a 580ms round trip delay ๐Ÿ›ฐ๏ธ and all values are in units of bytes, so no mystery numbers here ๐Ÿ”ฅ

TCP with Big Windows, No Naks:

               |---------------transfer rates----------------------|
   Window Size |  no error  |  10e-7 error rate | 10e-6 error rate |
   -----------------------------------------------------------------
     64K       |   94K      |      53K          |      14K         |
   -----------------------------------------------------------------
     72K       |   106K     |      51K          |      15K         |
   -----------------------------------------------------------------
     80K       |   115K     |      42K          |      14K         |
   -----------------------------------------------------------------
     92K       |   115K     |      43K          |      14K         |
    -----------------------------------------------------------------
     100K      |   135K     |      66K          |      15K         |
   -----------------------------------------------------------------
     112K      |   126K     |      53K          |      17K         |
   -----------------------------------------------------------------
     124K      |   154K     |      45K          |      14K         |
   -----------------------------------------------------------------
     136K      |   160K     |      66K          |      15K         |
   -----------------------------------------------------------------
     156K      |   167K     |      45K          |      14K         |
   -----------------------------------------------------------------
                                Figure 1.

TCP with Big Windows, and Naks:

               |---------------transfer rates----------------------|
   Window Size |  no error  |  10e-7 error rate | 10e-6 error rate |
   -----------------------------------------------------------------
     64K       |   95K      |      83K          |      43K         |
   -----------------------------------------------------------------
     72K       |   104K     |      87K          |      49K         |
   -----------------------------------------------------------------
     80K       |   117K     |      96K          |      62K         |
   -----------------------------------------------------------------
     92K       |   124K     |      119K         |      39K         |
   -----------------------------------------------------------------
     100K      |   140K     |      124K         |      35K         |
   -----------------------------------------------------------------
     112K      |   151K     |      126K         |      53K         |
   -----------------------------------------------------------------
     124K      |   160K     |      140K         |      36K         |
   -----------------------------------------------------------------
     136K      |   167K     |      148K         |      38K         |
   -----------------------------------------------------------------
     156K      |   167K     |      160K         |      38K         |
   -----------------------------------------------------------------
                                Figure 2.

Now the spicy part ๐Ÿ‘€ With a 10e-6 error rate, lots of naks and data packets got dropped, which is exactly why the transfer times swing all over the place ๐Ÿ“‰ Also, the machines used were SGI Iris 2500 Turbos running the 3.6 OS with the new TCP enhancements ๐Ÿ–ฅ๏ธ The Irises are slower than a Sun 3/260, but the Iris was used because of some source code restrictions ๐Ÿ™ƒ Initial results on the Sun showed slightly higher performance and less variance โœจ


Author's Address ๐Ÿ’Œ

Richard Fox

950 Linden #208

Sunnyvale, Cal, 94086

EMail: rfox@tandem.com ๐Ÿฆ‹

๐Ÿ“ฆ where this page comes from