RFC 1016

Something a Host Could Do with Source Quench: The Source Quench Introduced Delay (SQuID)

📅 published July 1987, from ietf

RFC 1016 🎀 Something a Host Could Do with Source Quench: The Source Quench Introduced Delay (SQuID) 💖

Network Working Group 💜 written by W. Prue and J. Postel 👑 from ISI, dated July 1987 📅. The header doesn't list a category, and it doesn't say it obsoletes or updates anything 🤷‍♀️.

Status of this Memo 🎀

okay so this memo is basically here to explore what a host could actually do with a source quench 🧐. The proposal is that each source host's IP module adds some delay between the datagrams it sends to the same destination host ⏳. The memo literally calls itself a "crazy idea paper" 🙈 and says discussion is essential, so bring your thoughts 💅. Distribution of this memo is unlimited 🌈.

Introduction ✨

so a gateway might throw away Internet datagrams if it doesn't have the buffer space to queue them for output to the next network on the route to the destination network 😬. When that happens, it can send a source quench message to the Internet source host of that datagram. A destination host can also send a source quench message if datagrams show up too fast to be processed 🏃‍♀️. A source quench message is basically a request for the host to cut back how fast it's sending traffic to that Internet destination 🛑. The gateway may send one for every message it discards 🗑️.

When a source host gets a source quench, it should cut back the rate at which it sends traffic to the specified destination, and keep doing that until it stops getting source quench messages from the gateway 🐢. Then it can gradually speed back up, until the quenches start showing up again [1,2] 🚦.

The gateway or host may send the source quench message when it approaches its capacity limit rather than waiting until the capacity is exceeded. Which means the datagram that set off the quench might still get delivered 📦✨.

The SQuID Concept 🦋

picture the IP module at the datagram source with a queue of datagrams to send. It has a parameter called "D" 🧚. D is the introduced delay between sending datagrams from the queue to the network. So when the IP module notices a datagram waiting to go to the network, it sends that datagram, then waits time D before even looking at the queue again ⏳. Normally, D is zero 💜.

Now imagine a source quench comes in (or any other signal says the host should slow down its transmissions to the network). That's when D goes up 📈. As time goes by, D goes back down 📉.

The SQuID Algorithm 🔥

this is the math bit, and it's not that scary, I promise 💅

          on increase event:

               D <-- maximum (D + K, I)
                                        (where K = .020 second,
                                               I = .075 second)

          on decrease event:

               D <-- maximum (D - J, 0)
                                        (where J = .001 second)

so on an increase event, D gets set to the maximum of D plus K and I, where K is .020 second and I is .075 second ⚡. On a decrease event, D gets set to the maximum of D minus J and 0, where J is .001 second 🌸.

An increase event is when one or more source quenches get received in an event period E, where E is 2.000 seconds ⏰. A decrease event is when S time has passed since D was decreased and there's a datagram to send, with S being 1.000 seconds ⏱️.

The host keeps a cache of D's for the last M hosts it communicated with 🗂️. If no datagrams go to a destination for some time, that destination's D is not decreased 🙅‍♀️. But if a destination isn't used for a long time, its D might fall out of the cache 👋.

some ideas for leveling this up 💫:

give each destination host, local subnet, or network its own outgoing queue of datagrams 📬
keep the cache of D's per network or local subnet, instead of per host 🏘️
"I" could be based on the basic speed of the slowest intervening network (see Appendix A) 🐢
"D" could be limited so it never goes below "I" if the refinement above gets implemented 🧱
"S" could be based on the round trip time ⏱️
"D" could be adjusted datagram by datagram based on the length of the datagrams, so you wait longer after a long datagram 📏
the delay algorithm could be implemented so that if a source doesn't send a datagram when it is next allowed (the introduced delay interval), or doesn't send for N such intervals, it gets a credit for one and only one free (no delay) datagram 🎁

Description of the Test Model ✨

okay so we needed a way to test our algorithm and all its different parameters 🧪. It was super important to check how IP running the SQuID algorithm interacts with TCP. We also wanted to try lots of combos of retransmission strategy and source quench strategy, which meant we needed total control over the entire test network. So we decided to build a whole Internet model 🌐

Here's the example setup we used to picture it:

 _______    LAN       _______     WAN      _______     LAN      _______
|   1   |            |   2   |            |   3   |            |   4   |
|TCP/IP |---10 Mb/s--|  IP   |---56 kb/s--|  IP   |---10 Mb/s--|TCP/IP |
|_______|            |_______|            |_______|            |_______|

A program written in C created queues, plus the structures that sit on them, representing datagrams carrying data, acknowledgments, and SQs 💖. It moved datagrams from one queue to the next based on the rules below 👇

A client fed TCP in node 1 data at whatever rate it would accept 🍓. The TCP function in node 1 chopped that data into fixed 512 byte datagrams to hand down to the IP in node 1. When a datagram was handed to IP for transmission, it got a timestamp, and a copy went onto a TCP ack-wait queue (data sent but not yet acknowledged) 🕰️. TCP treated data as sent the moment it handed it to IP, for retransmission timeout purposes, even though our algorithm has IP add delay before transmission. Main character drama, honestly 🙄

Each IP node had one queue in each direction, left and right 🧭. Each IP forwarded datagrams at the speed of the communications line going to the next node. So the fifth datagram on IP 2's queue going right would take 5 X 73 msec, or 365 msec, before it showed up at the end of IP 3's queue ⏳. We figured the time to process each datagram was less than the time it took to send it over the 56 kb/s lines, so it happened during those transmission times and wasn't included in the model. For the LAN communications that's not the case, but since they weren't the bottleneck on the path, we ignored their processing time 🙃. Because LAN bandwidth is usually shared, we treated the LAN bandwidth available to each IP instance as 1 Mb/s, which is a crude approximation 🫠

When data reached node 4, it was immediately given to the TCP receive function, which validated the sequence number ✅. If the datagram was in sequence, it got turned into an ack datagram and sent back to the source. An ack datagram carries no data and moves the right edge of the window, the window size, past the just acked data sequence number 🪟. The ack datagram is assumed to be 1/8 the length of a data datagram, so it can move from one node to the next 8 times faster 🚀. If the sequence number is less than expected (a retransmission because an ack got missed), it also gets turned into an ack. A datagram with a larger sequence number is queued indefinitely until the missing datagrams arrive 🫶

We also modeled the gateway source quench algorithm 🎀. When a datagram was put on an IP queue, the number on the queue was compared to an SQ keep level (SQK). If it was greater, an SQ was generated and returned to the sender. If it was larger than the SQ toss (SQT) level, the datagram was also discarded. Once SQs were generated, they kept being sent until the queue level dropped below SQ Low Water (SQLW), which sat below the original SQK level 📉. These percentages were modifiable, as were many parameters. An SQ could be lost if it exceeded the maximum queue size (MaxQ), but a source quench was never sent about tossing a source quench 🙈

Upon each transition from one node to the next, the datagram was vulnerable to loss due to errors 💔. The loss rate could be set as M losses out of N datagrams sent, so the model allowed for multi-datagram loss bursts or single datagram losses. For much of our testing we used a single datagram loss rate of 1 lost datagram per 300 datagrams sent. That might seem low for Internet simulation, but remember it doesn't include losses due to congestion 🤷‍♀️

Some of the network parameters we used:

a maximum queue length of 15 datagrams per IP direction, left and right
we started sending SQ if the queue was 70% full (SQK level)
we tossed data datagrams, but not SQ datagrams, if 95% of the queue was reached (SQT level)
we stopped SQing when a 50% SQLW level was reached (see above) ✨

We ignored additional SQs for 2 seconds after receiving one SQ 🛑. Why? Some Internet nodes only send one SQ for every 20 datagrams they discard, even though our model sent an SQ for every datagram discarded. Other IP nodes may send one SQ per discarded packet. The SQuID algorithm needed a way to handle both kinds of SQ generation 🤔. So we treated one SQ, or a burst of them, as a single event, and incremented our D by a larger amount than would be appropriate for responding individually to the multiple SQs from the verbose nodes 💅

The simulation did not fragment datagrams 🧩. Silly window syndrome was avoided ✨. The model did not implement or simulate the TTL (time-to-live) function ⏱️

The model allowed for a flexible topology definition, with many TCP source/destination pairs on host IP nodes or gateway IP nodes, with various windows allowed 🗺️. An IP node could have any number of TCPs assigned to it. Each line could have its own individually set speed. Any TCP could send to any other TCP. The routing from one location to another was fixed, so datagrams did not arrive out of sequence. However, datagrams arrived in ascending order, but not consecutively, on a regular basis because of datagram losses 📦. Datagrams going "left" through a node did not affect the queue size, or SQ chances, of data going "right" through the node 👈👉

The TCP retransmission timer algorithm used an Alpha of .15 and a Beta of 1.5 ⏲️. The test was run without the benefit of the more sophisticated retransmission timer algorithm proposed by Van Jacobson [5]

The program could display the queue sizes of the various IP nodes and the TCP under test as time passed, or do a crude plot of various parameters of interest, including SRTT, perceived round trip time, throughput, and the critical queue size 📊

As we watched the effects of different algorithms for responding to SQ, we adapted our model to react to SQ better 🔄. Early tests showed that if we incremented slowly and decremented quickly, we saw oscillations around the correct value, but more time was spent overdriving the network (and losing datagrams) than at a value that actually helped the congestion situation 😩

A significant problem is the delay between when some intermediate node starts dropping datagrams and sending source quenches, and when those source quenches arrive at the source host and can start affecting behavior at the data source ⏳. Because of this, and because an IP might send only one SQ for each 20 datagrams lost, we decided the increase in D per source quench should be substantial (for example, D should increase by 20 msec for every source quench), and the decrease with time should be very slow (for example, D should decrease 1 msec every second) 🐢. Note that this is the opposite behavior from what an early draft by one of the authors suggested 🙃

However, when many source quenches are received in a short time (for example, when a source quench is received for every datagram dropped), D increases excessively 😵. To keep D from growing too large, we decided to ignore subsequent source quenches for a time (for example, 2 seconds) once we had increased D 🛑

Tests were run with only one TCP sending data, so we could learn as much as possible about how an unperturbed session might run 🧪. Other test runs introduced and eliminated competing traffic dynamically between other TCP instances on the various nodes, to see how the algorithms reacted to changes in network load 🎢. A potential flaw in the model is that the defined TCPs with open windows always tried to forward data. Their clients feeding them data never paused to think about what they were going to type, never got swapped out in favor of other applications, and never turned the session around to listen to the other end for more user commands 💀. In other words, all of the simulated TCP sessions were doing file transfers 📁

The model was defined to allow many mixes of competing algorithms for responding to SQ 🥳. It allowed comparing effective throughput between TCPs with small windows and large windows, those whose IP would introduce inter-datagram delays, and those who totally ignored SQ. It also allowed comparisons with various inter-datagram increment amounts and decrement amounts. Because there were so many possible configurations and parameter combinations, only a few combinations were tested. It is hoped they were the most appropriate ones to concentrate on 🙏

Observed Results ✨

okay so, every single one of our algorithms oscillates 🌀 some way more dramatically than others 💀

If we add just the right amount of introduced delay, we seem to get the best throughput 🫶 but finding that sweet spot is not easy 🙈

Throughput takes a huge hit from even a single lost datagram 💔, at least for the short term. Picture this: a window that's 35 datagrams wide, an average round trip delay of 2500 msec, 512 byte datagrams, and one datagram lost right at the beginning 😱. TCP hands all 35 datagrams down to IP and starts a timer on the first one. Since that first datagram is missing, the receiving TCP won't send an acknowledgment, but it buffers all 34 of the out-of-sequence datagrams that did arrive. After 1.5 X 2500 msec, which is 3750 msec, the datagram times out and gets resent. It arrives and gets acked along with the other 34, 2500 msec later. Before the loss, we might've been sending at roughly the rate a 56 kb/s line could take, about one every 75 msec. After the loss, we're sending one in 6250 msec, which is over 83 times slower 😭

If the lost datagram is anything other than the first one, it's the exact same story once everything before it gets acked 🔁 So this example holds true for any single lost datagram in the window 💅

When SQ doesn't always cause datagram loss, the sender keeps sending too fast and the queue size oscillates a lot 🌊 It's super important for SQ to feed back into the sending system as soon as possible 🚨 so when the source host IP receives an SQ, it has to adjust the send rate for the datagrams still sitting on the send queue, not just the datagrams IP gets asked to send after the SQ arrives 🧚

Network delay goes up as the network queue lengths go up 📈 it's giving domino effect fr

Window size affects the chance of getting SQed 🎀 Look at our model above with a queue level of 15 for node 2 before any SQs get generated, and a window size of 20 datagrams. We assumed we could send data over the LAN at a sustained average rate of 1 Mb/s, which is about 18 times as fast as over the WAN ⚡ When TCP sends a burst of 20 datagrams to node 1, they make it to node 2 in 81 msec. The transition time from node 2 to node 3 is 73 msec, so in 81 msec, only one datagram gets forwarded to node 3. Thus the 17th, 18th, 19th, and 20th datagrams are lost every single time we send a whole window 💀 More get lost when the queue isn't empty.

Here's the tea ☕ If a sequence of acks comes back in response to the data we sent, the acks tend to return at the rate the data can traverse the net. That paces new send data by opening the window at the rate the network can actually accept it 🦋 However, the second one datagram gets lost, all of the subsequent acks get deferred and batched until the missing data block shows up. That one ack then acks all of the datagrams and opens the window back up to 20 🙃 This causes the max queue size to be exceeded all over again.

⚠️ one lost datagram is the villain of this whole story, and it keeps coming back for more 😤

If we assume a window smaller than the max queue size in the bottleneck node, then any time we send a window's worth of data, it goes out independently of the current size of the queue 🧃 The bigger the send window, the bigger the percentage of the stressed queue we send. If we send 50% of the stressed queue size any time that queue is more than 50% full, we threaten to overflow it 😬 Evenly spaced single datagram bursts have the least chance of overflowing the queue, since they represent the smallest percentage of the max queue you could possibly send 🌸

When a big window opens up (like when a missing datagram at the head of a 40 datagram send queue gets retransmitted and acked), the perceived round trip time for the datagrams sent afterward hits a minimum value and then goes up linearly 📉📈 The SRTT goes down and then back up in a nice smooth curve 🌈 This happens because IP won't add delay if the queue is empty and IP hasn't sent any datagrams to the destination for our introduced delay time. But when lots of datagrams get added to the IP pre-staged send queue in bursts, they all have the same send time as far as TCP is concerned 🫠 IP delays each datagram at the head of the queue by the introduced delay amount. The first one may go undelayed, like we just talked about, but all of the others are delayed by their ordinal number on the queue times the introduced delay amount 🍓

Here's a race I find lowkey wild 🏁 When one TCP session delays sending to IP and another one doesn't, the delayer gets better throughput because fewer datagrams get lost 🙌 The send window can also be increased to keep the pipeline full. But if the non-delayer uses windowing to reduce the chance of SQ datagram loss, their throughput might actually come out better, because no fair queuing algorithm is in place 🤔

If gateways send SQs early, don't toss data until it's critical, and keep sending SQs until a low water mark is hit, effective throughput seems to go up 🚀

At the startup of our tests, throughput was very high, then it dropped off quickly as the last of the window got clobbered 😵 Our model should have used a slow start up algorithm to minimize that startup shock. However, the learning curve to estimate the proper value for D was probably quicker 💜

A big chunk of the perceived RTT comes from the delay of getting off the TCP2IP (TCP transitional) queue when we used large windows 🐢 If IP invoked some back-pressure to TCP in a real implementation, this could be significantly reduced. Reducing the window would also do this for us, but at the expense of throughput 💅

After an SQ burst tosses datagrams, the sender gets stuck in a mode where TCP may only send one or two datagrams per RTT, until the queued but not acked segments fall back into sequence and get acked 😩 This assumes only the head of the retransmission queue gets retransmitted on a timeout. We can send one datagram upon timeout. When the ack for that retransmission comes back, the window opens up enough to send a second one. Then we wait for the next lost datagram to time out. Slow and painful, bestie 🐌

If we stop sending data for a while but allow D to be decreased, our algorithm makes the introduced delay dwindle away 🫧 We'd then go through a brand new startup learning curve and network oscillation sequence all over again 🔄

One thing we didn't see very often was TCP timing out a segment before the source IP even sent the datagram the first time 🙈 Like we discussed above, the first datagram on the queue of a large burst is delayed minimally, and the succeeding datagrams have linearly increasing delays. The smoothed round trip delay algorithm gets a chance to adapt to those perceived increasing round trip times 🧠✨

Unstructured Thoughts and Comments ✨

okay so the further a datagram travels down a route before it gets clobbered, the more network resources get wasted on it 😭 SQs that don't destroy the datagram they're talking about are way better than the ones that do, as long as return path resources are available 💅

any fix has to be implementable piecemeal 🧩 it can't be installed in all or most nodes at once. SQuID hits that requirement ✅ you could implement it, install it in one location, and it would still work effectively

ngl, if we can show that the new algorithm boosts effective throughput over implementations that don't have it, that might be the push vendors need to actually build it 🔥

once a source host has an established average minimum inter-datagram delay to a destination (see Appendix A), it should save that info across system restarts 💾 that number might get reused every time data goes to that host as the minimum inter-datagram delay 🎀

window closing algorithms lower the average inter-datagram delay and the burst size, but they don't touch the minimum inter-datagram spacing by TCP 🙈

okay this one is important 🚨 an IP gateway can tell it's on a critical path because its queues stay high or keep building up. its optimum queue size is one, because it always has something to do and the through node delay is at a minimum. it's super important that the gateway on the critical path doesn't discourage data flow so much that its queue drops to zero 😭 if the gateway tosses datagrams, that stops data flow for TCP for a while (as described in Observed Results above). this is why we want the gateway algorithm described above, the one that SQs but doesn't toss datagrams unless it's necessary 🙅‍♀️ optimally we should aim for a queue size somewhat larger than 1 but less than say 50% of the max queue size. large queues lead to large delay, and nobody wants that bestie 💀

TCP's SRTT gets artificially inflated 🎈 by the delay IP adds, but the perceived round trip time variance is probably smaller, which lets you use a smaller Beta value for the timeout value 🧠

so a decrease timer isn't needed for the "D" decrease function 🙌 upon the next sent datagram to a delayed destination, just decrease the delay by the amount of time since we last did this divided by the decrease timer interval. an alternate algorithm would be to decrease it by only one decrease unit amount if more than the timer interval has gone by. this fixes the problem where the delay, "D", dwindles away if we stop sending for a while. the longer we send using this "D", the more likely it is that it's too large a delay, and the more we should decrease it ✨

it's better for the network and the sender if our introduced delay is a little on the high side 🫶 it minimizes the chances of getting a datagram clobbered by sending it into a congested gateway. the lost datagram scenario described above showed that one lost datagram can reduce our effective delay by one to two orders of magnitude, temporarily 😱 also, if the delay is a little high, the net is less stressed and the queues get smaller, which reduces through network delay 🌈

the RTT experienced at a given time versus the minimum RTT possible for the given route does give a good measure of congestion 👀 if we ever get congestion control working, RTT may have little to do with the amount of congestion. effective throughput compared with the possible throughput (or some other measure) is the only real measure of congestion 💖

slow startup of TCP is a good thing, and it should be encouraged as an additional mechanism for easing network overload 🐢💜

the network dynamics tend to bunch datagrams 🫠 if we properly space data instead of bunching it up like windowing techniques do to control queue overflow, we get more throughput, because the absolute rate we can send is set by our pacing, not the RTT. that way we eliminate "stochastic bunching" [6] 🦋

the longer the RTT, the more network resources the data takes to traverse the net 🛣️

should "fair queuing" say that a longer route transfer should get less band width than a shorter one (since it eats more of the net)? being fair locally on each node might be unfair overall to datagrams that cross lots of nodes 🤔

if we solve congestion problems today, we'll start loading up the net with more data tomorrow 📦 when that causes congestion in a year, will that kind of congestion be harder to solve than today's, or is it just not our problem? John Nagle suggests:

"In a large net, we may well try to force congestion out to the fringes and keep the interior of the net uncongested by controlling entry to the net. The IMP-based systems work that way, or at least used to. This has the effect of concentrating congestion at the entrance to the long-haul system. That's where we want it; the Source Quench / congestion window / fair queuing set of strategies are able to handle congestion at the LAN to WAN bottleneck [7]."

our algorithm should try to push network congestion out to the extremities and keep the interior network congestion free 🫧

using the algorithm is aesthetically appealing, honestly 😍 the data sits in our local queue instead of eating resources inside the net. we only give data to the network when it's ready to accept it 🎀

an averaged minimum inter-datagram arrival value gives a measure of the network bottleneck speed at the receiver 📏 if the receiver doesn't defer or batch together acks, the same thing can be learned from the inter-datagram arrival time of the acks. the catch is that IP doesn't know the datagram contents. however, IP does know which host a datagram comes from 🧚

if SQuID limits the size of its pre-net buffering properly (causing back-pressure to TCP), then artificially high RTT measurements won't happen 🙌

someday TCP might get a way to query IP for the current introduced delay, D, for a given destination, and if the value is too excessive, abort or not start a session 🚫

with the new algorithm, TCP could have an arbitrarily large window to send into without fear of overrunning queue sizes in intermediate nodes (not that any TCP ever considered having that fear before 💅). so it could have a window size that lets it always be sending, keeping the pipe full and seldom getting stuck in stop-and-wait mode 🌊 this presupposes that the local IP can cause some sort of back pressure so the local IP's queues aren't overrun. TCP would still be operating in burst mode, but the local IP would send a datagram for TCP as often as the network can accept it, keeping the data flow continuous, though potentially slow 🐌💕

experience implementing protocols suggests avoiding timers in protocols whenever possible ⏱️ IP, as currently defined, doesn't use timers. the SQuID algorithm uses two at the IP level 😬 one way to eliminate the introduced delay decrease timer is to decrease the D value whenever we check the send queue for data to send. we would decrease "D" by one "J" unit if "S" time, or more, has elapsed. the other timer isn't so easily eliminated. if the IP implementation is periodically awakened to check for work to do, it could check the time stamps at the head of the queues to see if any datagrams have waited long enough. this means we'd necessarily wait too long before sending, but it may not be too far off from our desired rates. the extra delay would help congestion and reduce our chances of SQ 🙏 another option is setting a real timer that's say 25-50% too large, and hoping our periodic work in IP will let us notice a datagram is delayed enough to send it. when we send the datagram, we cancel the timer, avoiding the timer interrupt and environment swap ✨ in other implementations, the communications interface or the link level protocol may be able to provide the timing needed without interrupts to the main processor 🧚‍♀️

background tasks like some file transfers could query IP for the latest delay characteristics for a given destination 📥 that way they can decide whether it's appropriate to use network resources now or whether it'd be better to wait until later 🕰️

we should consider what happens if IP (using the SQuID algorithm) and TCP both introduce delay between datagrams 🤯 if TCP's delay is greater than IP's, then when IP gets the datagrams, it forwards them immediately, as described in the algorithm above. that's because when the IP send queue is empty and it has been at least as long as IP wants, the higher level protocol, TCP, gets one free (no delay) send 🎁 note also that IP will decrease the amount of delay it wants introduced because of the successful transmissions without SQs. this would affect other protocols that might also send to the same destination 👯. if TCP sends too quickly, IP will protect the network from its indiscretion, as described in the basic algorithm. however, TCP's round trip time estimates will be much closer to reality 🎯 which means a lost datagram gets detected more quickly. if TCP also uses windowing to limit its sending rate, it might, because of its success with a smaller window, increase the window size, which increases TCP's efficiency 📈

as this algorithm gets implemented everywhere, the SQ Keep (SQK) and SQ Low Water (SQLW) queue level percentages should be dropped to reduce queue sizes and thus the through net delay 📉 the percentage we lower SQK and SQLW to should be adjusted based on the percentage of time the queue is empty. the more the queue is empty, the more likely it is that the node is discouraging data flow too much 🥺 the more time the queue is not empty but not too full, the more likely it is the node isn't excessively discouraging data flow 💅 how uniform the queue size is tells us how well the network citizens are behaving 🏙️✨

as congestion gets pushed to the sources, gateways that are bottlenecks can more easily detect someone not playing by the rules (sending datagrams in bursts) and deal with the offender 🚔💜

Appendix A -- Determination of the Value for the Parameter "I" ✨

okay so when something happens and the delay we're using is already super tiny, we need a first guess fast 🏃‍♀️. The move is to use the transmission time for an average-sized datagram over the slowest communications link. How a real IP node is supposed to guess that value gets talked about below 👀. In our example, the bottleneck is the hop between node 2 and node 3. On the 56 kb/s line, a 512 byte datagram takes 73 msec, with no queuing or processing time considered. We define that as the MIAT (minimum inter-datagram arrival time) 💅. Like, in a perfect network that only cares about transmission speed, that's the least amount of time you'd expect between datagrams showing up at the destination, because the bottleneck slows everything down. Just know this stops being true if the datagrams don't all take the same path 🙈.

The MIAT, minimum inter-datagram arrival time, can be guessed at by comparing the arrival timestamps of consecutive datagrams from a source of data. Every value you're considering needs to be adjusted up or down based on the size of the second datagram received compared to the typical datagram size. Simply put, a datagram that's half the size of the average one can cross a line in half the time 🍓. So you double its IAT before you consider it for an MIAT. If the timestamp is taken based on an event triggered by the start of the datagram arriving, rather than the completion of it arriving, then the first datagram's size is the limiting length and should be used to adjust its IAT. To keep the algorithm simple, arrival times for short datagrams could be ignored, and so could IATs that are orders of magnitude too large (see below) 🧚.

Once a minimal value is found by looking for small values over a minute or more, that value might be time averaged with a value kept like TCP's SRTT, so the effects of a false MIAT get reduced 🌸. We could assume the limiting facility is a 9.6 kb/s line, a 56-64 kb/s line, or a 1.5 Mb/s line. These facilities would give a MIAT of 427 msec, 73-64 msec, or 3 msec respectively, for a datagram 512 bytes long. The gaps between those are almost orders of magnitude, lowkey wild 😭. If the MIAT a node measures isn't in one of these ranges but is close, the value it's closest to may be used as its MIAT from that source ✨.

One of the best things about this measurement is that it's an entirely passive measurement. No additional traffic is needed to measure it 🦋. If a source isn't sending data continuously, then the longer measured values won't be validated as minimal values. The assumption is that at least sometimes the source will send multiple datagrams at a time 💜.

The MIAT measurement is totally unaffected by the satellite delay characteristics of any intervening facilities 🛰️. The chance of getting a valid minimal reading is affected by the number of nodes traversed, but the value measured, if it's valid, will not be affected by the number of nodes traversed. Stated another way, when a pair of datagrams travels from one node to the next, they can be separated by a datagram from another source, and that's a whole mood. Both of these factors affect SRTT. The value obtained requires no topological knowledge of the route 🗺️.

A potential problem with the measurement is that it won't be the proper value for some forms of alternate routes 🙃. If a T1 link is the bottleneck route sometimes, and other times it's a 56 kb/s link, our first guess for inter-datagram delay would be too small for the 56 kb/s line route. Another problem is that if one datagram goes via one route and the next goes via another, their inter-datagram arrival difference could lead to a small false measurement. If intervening networks fragment datagrams, then the measured IAT between segments could be misleading. A solution to this problem is to ignore fragmented datagram IATs 💀.

This number represents the minimum inter-datagram delay the sending IP should use to send to us, the measuring site, for the given datagram size. If we assume the return path goes through the same facilities or the same type, then, as described above, this value can be the first guess for the inter-datagram introduced delay, "D" (in the algorithm). It represents the "I" parameter 👑.

These MIATs may be cached on a host, subnet, or network basis. The last "n" hosts' MIATs could be kept. For infrequent destinations, an entry per destination network would apply to many destinations. If the local net is actually a subnet, then the other local subnet MIATs could be kept 🎀.

If a good algorithm is found for MIAT, comparing it to the average IAT (during data transfer) would give a good measure of the amount of network traffic load 📈. Since IP has no idea when the source of data is sending as fast as possible, upper layer protocols would have to figure that out for themselves to get a valid average. IP could, however, provide an interface to get the current MIAT for a given destination 🫶.

References 📚

[1] Postel, Jon, "Internet Protocol - DARPA Internet Program Protocol Specification", RFC 791, ISI, September 1981.

[2] Postel, Jon, "Internet Control Message Protocol - DARPA Internet Program Protocol Specification", RFC 792, ISI, September 1981.

[3] Karels, M., "Re: Source Quench", electronic mail message to J. Postel and INENG-INTEREST, Tue, 24 Feb 87.

[4] Nagle, John B., "On Packet Switches With Infinite Storage", RFC 970, FACC Palo Alto, December 1985.

[5] Jacobson, Van, "Re: interpacket arrival variance and mean", electronic mail message to TCP-IP, Mon, 15 Jun 87 06:08:01 PDT

[6] Jacobson, Van, "Re: Appropriate measures of gateway performance" electronic mail message to J. Noel Chiappa and INENG-INTEREST, Sun, 22 Mar 87 15:04:44 PST.

[7] Nagle, John B., "Source quench, and congestion generally", electronic mail message to B. Braden and INENG-INTEREST, Thu, 5 Mar 87 11:08:49 PST.

[8] Nagle, John B., "Congestion Control in IP/TCP Internetworks", RFC 896, FACC Palo Alto, 6 January 1984.

📦 where this page comes from

ietf: rfc1016 1987-07
💖 girlified by claude-haiku-5-5 on 2026-10-15: a language model's rewrite of the original, under its terms