RFC 7893

Pseudowire Congestion Considerations

๐Ÿ“… published June 2016, from ietf

Pseudowire Congestion Considerations ๐ŸŽ€

Okay bestie, let's get into RFC 7893! ๐Ÿ’– It's an Informational doc (so not a Standards Track thing, just vibes and analysis ๐ŸŒธ) by Y(J) Stein of RAD Data Communications, D. Black of EMC Corporation, and B. Briscoe of BT. It came out in June 2016 and it's giving ISSN 2070-1721 energy. It doesn't obsolete or update any other RFCs. ๐Ÿซถ

Abstract

Pseudowires (PWs) are everywhere now as a way to tunnel traffic ๐Ÿš‡, and they can end up in unmanaged setups fighting for network resources with other PWs AND with regular non-PW traffic like TCP/IP flows ๐Ÿ˜ญ. So it's worth figuring out when that competition is actually fine, meaning the PW traffic doesn't seriously hurt other traffic or contribute more congestion than it should. โœจ

The big conclusion: PWs carrying responsive traffic behave the way we want without needing extra mechanisms ๐Ÿ’…. For inelastic PWs, like Time Division Multiplexing (TDM) PWs, they derive a bound where the PW uses no more network capacity than a TCP flow ๐Ÿ”ฅ. For TDM PWs, the congestion level where the PW can no longer deliver acceptable TDM service is never significantly higher, and is usually way lower, than that bound ๐Ÿ™ˆ. So as long as the PW shuts down when it can't deliver acceptable TDM service, it never does significantly more harm than even a single TCP flow ๐Ÿ‘‘. If TDM service doesn't shut down on its own, then a mechanism to block persistently unacceptable TDM pseudowires is required. โญ


Status of This Memo

This document is not an Internet Standards Track specification; it is published for informational purposes. ๐Ÿฆ‹

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 7841.

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc7893. ๐ŸŒˆ

Copyright Notice

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. ๐Ÿ’œ

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. ๐Ÿซถ

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   5
3.  PWs Comprising Elastic Flows  . . . . . . . . . . . . . . . .   6
4.  PWs Comprising Inelastic Flows  . . . . . . . . . . . . . . .   7
5.  Conclusions . . . . . . . . . . . . . . . . . . . . . . . . .  19
6.  Security Considerations . . . . . . . . . . . . . . . . . . .  19
7.  Informative References  . . . . . . . . . . . . . . . . . . .  19
Appendix A.  Loss Probabilities for TDM PWs . . . . . . . . . . .  22
Appendix B.  Effect of Packet Loss on Voice Quality for
             Structure-Aware TDM PWs  . . . . . . . . . . . . . .  23
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  27

1. Introduction ๐Ÿงš

1.1. What a PW even is

A pseudowire (PW) (see [RFC3985]) is a construct for tunneling a native service, like Ethernet or TDM, over a Packet Switched Network (PSN), like IPv4, IPv6, or MPLS. ๐Ÿ“ The PW packet wraps a unit of native service info by prepending the headers needed for transport in that particular PSN (which must include a demultiplexer field to tell the different PWs apart) and preferably the 4-byte Pseudowire Emulation Edge-to-Edge (PWE3) control word. ๐Ÿ’…

PWs have no bandwidth reservation or control mechanisms, which is the tea โ˜•. So when multiple PWs run in parallel, and/or in parallel with other flows, there's no defined way to allocate resources for any particular PW, or to stop a particular PW from hurting its neighbors. ๐Ÿ˜ญ The case where the service provider network gives a PW enough capacity is well understood, so we won't dig into it here. The real concern is when PWs share capacity with elastic or congestion-responsive traffic, whether the sharing was planned by a provider or happens because an end user deployed a PW. ๐Ÿ”ฅ

PWs are most often placed in MPLS tunnels, but this doc only covers PWs in IPv4 or IPv6 PSNs; MPLS PSNs are out of scope here. ๐Ÿ™ˆ There are several mechanisms for transporting PWs over an IP infrastructure, including:

  • UDP/IP encapsulations as defined for TDM PWs [RFC4553] [RFC5086] [RFC5087],
  • PWs based on Layer 2 Tunneling Protocol (L2TPv3) [RFC3931],
  • MPLS PWs directly over IP according to RFC 4023 [RFC4023], and
  • MPLS PWs over Generic Routing Encapsulation (GRE) over IP according to RFC 4023 [RFC4023].

Whenever PWs go over IP, they may compete for network resources with neighboring congestion-responsive flows (e.g., TCP flows). ๐Ÿฆ‹ In this doc, the authors study the effect of PWs on those neighbors and discover that the negative impact of PW traffic is generally no worse than that of congestion-responsive flows [RFC2914] [RFC5033]. โœจ

At first glance, you might think a PW over IP counts as a single flow, on par with a single TCP flow ๐Ÿค”. If we accepted that, we'd require a PW to back off under congestion to use no more bandwidth than a single TCP flow would under those conditions (see [RFC5348]). But since PWs can carry traffic from many users, it makes more sense to treat each PW as equivalent to multiple TCP flows. ๐Ÿ‘‘

The following two sections consider two types of PWs:

  • Elastic Flows: Section 3 concludes that the congestion response of a PW carrying elastic (e.g., TCP) flows is no different from the aggregated behaviors of the individual elastic flows, had they not been encapsulated within a PW. ๐ŸŒธ
  • Inelastic Flows: Section 4 considers inelastic constant bit rate (CBR) TDM PWs [RFC4553] [RFC5086] [RFC5087] competing with TCP flows. ๐Ÿ’– Such PWs require a preset amount of bandwidth, which may be lower or higher than what an otherwise unconstrained TCP flow would use under the same network conditions. ๐Ÿ˜ณ Either way, such a PW can't respond to congestion in a TCP-like manner; though, admittedly, the total bandwidth it consumes stays constant and doesn't increase to grab extra bandwidth as TCP rates back off. ๐Ÿง  For TDM services, the authors show that service quality degradation generally happens before the TDM PW becomes TCP-unfriendly. โญ For TDM services that don't automatically shut down when they persistently fail to meet acceptable TDM service criteria, a transport circuit breaker [CIRCUIT-BREAKER] may be employed as a last resort to shut down a TDM pseudowire that can no longer deliver acceptable service. ๐Ÿ›‘

So in both cases, pseudowires won't inflict significant harm on neighboring TCP flows ๐ŸŽ‰: in one case they respond adequately to congestion, and in the other they get shut down for being unable to deliver acceptable service before they can hurt their neighbors. ๐Ÿฅน

Note: This document contains a large number of graphs that are necessary for its understanding, but could not be rendered in ASCII. It is strongly suggested that the PDF version be consulted. ๐Ÿ“„

(Part 1 of 5 โ€” more coming, so stay tuned, bestie! ๐Ÿงšโ€โ™€๏ธ๐Ÿ’œ)

3. PWs Comprising Elastic Flows โœจ

okay so this section is about Ethernet PWs that mostly carry congestion-responsive traffic ๐Ÿซถ the stuff that backs off when the network gets crowded. the authors are building on the note in Section 8 (Congestion Control) of [RFC4553], and the big reveal is that the congestion avoidance behavior we want just happens automatically. no extra mechanisms needed ๐Ÿ’…

let's say an Ethernet PW is carrying a bunch of TCP flows, and it's flowing right alongside several regular TCP/IP flows. every Ethernet PW packet holds one Ethernet frame, which holds one IP packet, which holds one TCP segment. so if an intermediate router drops a packet to signal congestion, exactly one end-user TCP/IP packet gets dropped, whether or not it's wrapped up in the PW ๐ŸŽ€

the result is that the TCP flows inside the PW face the same drop probability as the TCP flows that aren't in a PW. so the sender's reaction (resending the packet and slowing its sending rate) is the same either way, whether the flow is running directly over IP or sitting inside the PW โœจ basically, individual TCP flows aren't rewarded or punished for riding a PW. ๐Ÿฆ‹

now, an elastic PW doesn't act like a single TCP flow. it eats the aggregated bandwidth of all its component flows. but if those component TCP flows back off by some percentage, the bandwidth of the PW as a whole drops by that very same percentage, purely because of how its component flows combine ๐Ÿ˜ญ it's giving teamwork.

and honestly? this is exactly the behavior we want ๐ŸŒธ if individual TCP flows got rewarded for being carried over a PW, people would be incentivized to build PWs for no operational reason at all. and if they got penalized, that would be a deterrent that could slow down pseudowire deployment. neither is it.

there have been proposals to bolt extra TCP-friendly mechanisms onto PWs, for example by carrying PWs over DCCP ๐Ÿ™ˆ but with everything above in mind, it's pretty clear that would squash the PW down to the bandwidth of a single flow, instead of N flows, and penalize the constituent TCP flows. plus the individual TCP flows would still back off, because their endpoints have no idea they're riding a PW. so the flow's throughput would get degraded compared to a non-PW-encapsulated flow, which goes against the behavior we want. ๐Ÿ’€

we've kept our focus on TCP traffic carried by Ethernet PWs, since those are by far the most commonly deployed packet-carrying pseudowires. but it's not overly difficult to show that the same result holds for other PW types too, like ATM or frame-relay pseudowires ๐Ÿงš

4. PWs Comprising Inelastic Flows ๐Ÿ’–

now we're getting into the spicier stuff ๐Ÿ˜ค inelastic PWs, like TDM PWs [RFC4553] [RFC5086] [RFC5087], are potentially more problematic than the elastic PWs from the last section. as noted in Section 8 (Congestion Control) of [RFC4553], TDM PWs are constant bit rate (CBR), so they can't respond to congestion incrementally in a TCP-like fashion. the flip side is that, being CBR, they also don't make things worse by grabbing extra bandwidth when neighboring TCP flows back off โœจ

since a TDM PW eats a constant amount of bandwidth, you might think that if it's endangering the whole network, the only option is to shut it down. but that would deny service to every customer of the TDM native service ๐Ÿ˜ญ thankfully, under certain conditions it may be possible to reduce the bandwidth consumption of an emulated TDM service ๐Ÿซถ a very common case is a TDM native service carrying voice channels that aren't all active at once. the ATM Adaptation Layer 2 (AAL2) mode of [RFC5087] (maybe along with connection admission control) can enable this bandwidth adaptation, but it comes at the cost of more sophisticated native service processing (NSP). ๐ŸŽ€

we're going to focus on structure-agnostic TDM PWs [RFC4553], though a similar analysis can be readily applied to structure-aware PWs (see Appendix B). the plot twist: for many cases of interest, a TDM PW, even when treated as a single flow, behaves reasonably without any additional mechanisms ๐Ÿ‘‘ we also show that at the level of congestion where a TDM PW can no longer deliver acceptable TDM service, a single unconstrained TCP flow would typically still consume more capacity than a whole TDM PW. ๐Ÿ”ฅ

so, to make sure a TDM PW doesn't inflict significantly more harm than a TCP flow, it's enough to shut down a TDM PW that's persistently unable to deliver acceptable TDM service ๐Ÿ™ˆ this shutdown could be handled by a managed transport circuit breaker, which means an automatic mechanism for terminating an unresponsive flow during persistently high levels of congestion [CIRCUIT-BREAKER]. the vibe is that a transport circuit breaker is a protection mechanism of last resort, just like an electrical circuit breaker only trips when it's absolutely necessary โšก

For the avoidance of doubt: the above does NOT mean a TDM PW should be shut down just because it becomes TCP-unfriendly. ๐Ÿšซ It only says that shutting down a TDM PW that can no longer deliver acceptable TDM service ensures the PW doesn't contribute to congestion significantly more than a TCP flow would.

also, being unable to deliver acceptable TDM service for a short amount of time is not enough justification to shut a TDM PW down ๐Ÿฅบ TCP flows react within a round-trip time, but service commissioning and decommissioning are generally time-consuming, so they should only happen when it's clear the congestion isn't transient ๐Ÿ“

to compare TDM PWs with TCP flows in a quantitative way, we'll look at the effect of TDM PW traffic next to TCP traffic that has the same packet size and delay. this might be an overly pessimistic comparison, since TDM PW packets are often configured to be short to minimize latency, while TCP packets are free to be much larger ๐Ÿ’…

two network parameters matter here: the one-way delay (D) and the packet loss ratio (PLR). the one-way delay of a native TDM service is the physical time-of-flight plus 125 microseconds for each TDM switch traversed, so it's very small compared to typical PSN network-crossing latencies ๐Ÿ˜ญ since TDM services are designed around this low latency, emulated TDM services are usually required to have similarly low end-to-end delay. in our comparisons, we only consider one-way delays of a few milliseconds โœจ

now, packet loss. the relevant RFCs specify actions to take when a lost packet is detected. structure-agnostic transport has no alternative but to output an "all-ones" Alarm Indication Signal (AIS) pattern toward the TDM circuit. when that lasts long enough, the receiving TDM device reads it as a fault indication (see Appendix A) ๐Ÿšจ TDM standards (such as [G826]) put strict limits on how many of these faults are tolerated. the calculations in Appendix A show that only loss probabilities in the realm of fractions of a percent are relevant for structure-agnostic transport ๐Ÿซ 

structure-aware transport regenerates frame alignment signals, so it avoids AIS indications caused by infrequent packet loss ๐Ÿ™Œ also, for TDM circuits carrying voice channels, packet loss concealment algorithms are possible (these have been described for TDM PWs before). but even structure-aware transport stops providing a useful service at about 2 percent loss probability. so in our comparisons, we'll only consider PLRs of 1 or 2 percent ๐ŸŽ€

TCP Friendly Rate Control (TFRC) [RFC5348] gives us a simplified formula for TCP throughput as a function of round-trip delay and packet loss ratio ๐Ÿงฎ

                                 S
    X     = ------------------------------------------------
              R  ( sqrt(2p/3) + 12 sqrt(3p/8) p (1+32p^2) )

where:

  • X is the average sending rate in bytes per second,
  • S is the segment (packet payload) size in bytes,
  • R is the round-trip time in seconds,
  • p is the packet loss probability (i.e., PLR/100).

now we can compare the bandwidth consumed by TDM pseudowires with that of a TCP flow, for a given packet loss ratio and one-way end-to-end delay (taken to be half the round-trip delay R). ๐Ÿฆ‹ the results are in the accompanying figures, which are only in the PDF version of this document ๐Ÿ“„

in Figures 1 and 2, we see the usual rate vs. packet loss plot for low-rate TDM (both T1 and E1) traffic, along with TCP traffic using the same payload size (64 or 256 bytes respectively). the TDM rates are constant (T1 and E1 have payload throughputs of 1.544 Mbps and 2.048 Mbps respectively), and Structure-Agnostic TDM over packet (SAToP) can only faithfully emulate a TDM service up to a PLR of about half a percent. so the T1 and E1 pseudowires show up as line segments on the graph ๐Ÿ“ˆ the TCP rate equation, meanwhile, produces rate curves that depend on both one-way delay and packet loss ๐ŸŒˆ

for large packet sizes, short one-way delays, and low packet loss ratios, TDM pseudowires typically consume much less bandwidth than TCP would under identical conditions โœจ for small packets, long one-way delays, and high packet loss ratios, TDM PWs potentially consume more bandwidth, but only marginally ๐Ÿ™ˆ also, our "apples to apples" comparison forced the TCP traffic to use packets smaller than would be typical ๐Ÿ“

similarly, in Figures 3 and 4 we repeat the exercise for the higher-rate E3 and T3 pseudowires (rates 34.368 and 44.736 Mbps respectively), allowing delays and PLRs suitable for those signals. the TDM pseudowires consume much less bandwidth than TCP here, for all reasonable parameter combinations ๐Ÿ‘‘

--------------------------------------------------------------------
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I             E1/T1 PWs vs. TCP for segment size 64B               I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                     (only in PDF version)                        I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
--------------------------------------------------------------------

          Figure 1: E1/T1 PWs vs. TCP for Segment Size 64B

Okay bestie, here come the figures! ๐Ÿ“Šโœจ Each one is a placeholder box that says "(only in PDF version)", so the actual charts live in the PDF. Here's the first one ๐ŸŽ€

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I             E1/T1 PWs vs. TCP for segment size 256B              I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

Figure 2: E1/T1 PWs vs. TCP for Segment Size 256B ๐ŸŒธ

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I             E3/T3 PWs vs. TCP for segment size 536B              I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

Figure 3: E3/T3 PWs vs. TCP for Segment Size 536B ๐Ÿฆ‹

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I             E3/T3 PWs vs. TCP for segment size 1024B             I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

Figure 4: E3/T3 PWs vs. TCP for Segment Size 1024B ๐Ÿงš

Now the good part ๐Ÿ”ฅ We can use the TCP rate equation to pin down the exact conditions where a TDM PW uses no more bandwidth than a TCP flow between the same endpoints under identical conditions. ๐Ÿ’… You swap the round-trip delay for twice the one-way delay D, set the bandwidth to the TDM service BW, and make the segment size the TDM fragment (counting the PWE3 control word too). Then you get this condition for a TDM PW:

              4 S
       D < -----------
             BW f(p)

where:

  • D is the one-way delay,
  • S is the TDM segment size (packet excluding overhead) in bytes,
  • BW is the TDM service bandwidth in bits per second,
  • f(p) = sqrt(2p/3) + 12 sqrt(3p/8) p (1+32p^2).

Okay, so this condition is basically a "friendly" operating envelope ๐Ÿซถ A TDM PW that eats no more bandwidth than a TCP flow causes no more congestion than that TCP flow does. So it's totally fine to park the TDM PW next to congestion-responsive traffic like TCP. โœจ But if the TDM PW ate way more bandwidth than a TCP flow, it could add to congestion out of proportion, and mixing it with congestion-responsive traffic might not be a good look. ๐Ÿ’€ Also, we're not getting into the whole debate about whether the TCP-friendliness concept is valid ๐Ÿ™ˆ We're just saying that a TDM PW is definitely acceptable if it causes no more congestion than a single TCP flow. No cap.

Two caveats, because this was derived under steady-state conditions ๐Ÿ“ First, the condition doesn't tell you what to do with a TDM PW that meets it at the start but then gets stuck in a worsening network. For that you'd also need to look at how quickly the responsive flows react to congestion events. Second, the derivation assumed the TDM PW was competing with long-lived TCP flows. That's because it made the comparison easy to quantify against something widely seen as a safe response to congestion. Short-lived TCP flows could end up at a disadvantage next to a long-lived TDM PW that meets the condition above. Not fair, but that's the math ๐Ÿ˜ญ

Now the results ๐Ÿ‘‘ Figures 5 and 6 show that TDM pseudowires carrying T1 or E1 native services meet the condition for every parameter of interest with large packet sizes (for example, S=512 bytes of TDM data). For the SAToP default of 256 bytes, as long as the one-way delay is under 10 milliseconds, the loss probability can go above 0.3 or 0.6 percent. For packets with 128 or 64 bytes, things get more troublesome, but there are still parameter ranges where the TDM PW consumes less than a TCP flow under similar conditions. โญ And Figures 7 and 8 show that E3 and T3 native services with the SAToP default of 1024 bytes of TDM per packet meet the condition across a broad spectrum of delays and PLRs. Slay ๐ŸŒˆ

Here are the compatibility-region figures ๐Ÿ“

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                    T1 compatibility regions                      I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

Figure 5: TCP Compatibility Areas for T1 SAToP ๐Ÿ’œ

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                    E1 compatibility regions                      I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

Figure 6: TCP Compatibility Areas for E1 SAToP ๐ŸŽ€

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                    E3 compatibility regions                      I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

Figure 7: TCP Compatibility Areas for E3 SAToP ๐Ÿฆ‹

   --------------------------------------------------------------------
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                    T3 compatibility regions                      I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                     (only in PDF version)                        I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   I                                                                  I
   --------------------------------------------------------------------

              Figure 8: TCP Compatibility Areas for T3 SAToP

5. Conclusions โœจ

okay so the figures from the last bit show that TDM service quality starts to degrade before a TDM PW would eat up more bandwidth than a comparable TCP flow ๐Ÿ’… way to be sneaky, TDM ๐Ÿ™ˆ so yeah, TDM PWs can't back off the way TCP does, but the ones that actually deliver acceptable TDM service don't contribute to congestion way more than a TCP flow would ๐Ÿซถ

and combining that with what we figured out earlier, that Ethernet PWs automatically respond in a TCP-like fashion (see Section 3) โœจ, our big takeaway is that PW-specific congestion-avoidance mechanisms are generally not required ๐ŸŽ€ this is true even for TDM PWs, as long as the TDM management plane shuts the service down when its parameters stay below the levels the relevant TDM standards want ๐ŸŒธ

If the TDM service doesn't automatically shut down, then a mechanism to block persistently unacceptable TDM pseudowires is required ๐Ÿ’€ or, as a last resort, a transport circuit breaker [CIRCUIT-BREAKER] may be triggered. bestie, do not let your broken pseudowire just run wild ๐Ÿ˜ญ

6. Security Considerations ๐Ÿ”

this document doesn't introduce any new congestion-specific mechanisms ๐Ÿ™…โ€โ™€๏ธ so it doesn't add any new security considerations beyond the ones that already exist for PWs in general ๐Ÿฆ‹ not a new threat, just the usual suspects ๐Ÿ’œ

7. Informative References ๐Ÿ“š

[CIRCUIT-BREAKER] Fairhurst, G., "Network Transport Circuit Breakers", Work in Progress, draft-ietf-tsvwg-circuit-breaker-15, April 2016.

[G703] ITU-T, "Physical/electrical characteristics of hierarchical digital interfaces", ITU Recommendation G.703, April 2016.

[G775] ITU-T, "Loss of Signal (LOS), Alarm Indication Signal (AIS) and Remote Defect Indication (RDI) defect detection and clearance criteria for PDH signals", ITU Recommendation G.775, October 1998.

[G826] ITU-T, "Error Performance Parameters and Objectives for International Constant Bit Rate Digital Paths at or above Primary Rate", ITU Recommendation G.826, December 2002.

[P50App1] ITU-T, "Telephone Transmission Quality, Telephone Installations, Local Line Networks: Appendix 1", ITU-T Recommendation P.50, February 1998.

[P800] ITU-T, "Methods for subjective determination of transmission quality", ITU Recommendation P.800, June 1998.

[P862] ITU-T, "Perceptual evaluation of speech quality (PESQ): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs", ITU Recommendation P.826, February 2001.

[PACKET-LOSS] Stein, J(Y). and I. Druker, "The Effect of Packet Loss on Voice Quality for TDM over Pseudowires", Work in Progress, draft-stein-pwe3-tdm-packetloss-01, December 2003.

[RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D., and P. Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, DOI 10.17487/RFC2784, March 2000, <http://www.rfc-editor.org/info/rfc2784>.

[RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, DOI 10.17487/RFC2914, September 2000, <http://www.rfc-editor.org/info/rfc2914>.

[RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, DOI 10.17487/RFC3031, January 2001, <http://www.rfc-editor.org/info/rfc3031>.

[RFC3931] Lau, J., Ed., Townsley, M., Ed., and I. Goyret, Ed., "Layer Two Tunneling Protocol - Version 3 (L2TPv3)", RFC 3931, DOI 10.17487/RFC3931, March 2005, <http://www.rfc-editor.org/info/rfc3931>.

[RFC3985] Bryant, S., Ed. and P. Pate, Ed., "Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture", RFC 3985, DOI 10.17487/RFC3985, March 2005, <http://www.rfc-editor.org/info/rfc3985>.

[RFC4023] Worster, T., Rekhter, Y., and E. Rosen, Ed., "Encapsulating MPLS in IP or Generic Routing Encapsulation (GRE)", RFC 4023, DOI 10.17487/RFC4023, March 2005, <http://www.rfc-editor.org/info/rfc4023>.

[RFC4553] Vainshtein, A., Ed. and YJ. Stein, Ed., "Structure-Agnostic Time Division Multiplexing (TDM) over Packet (SAToP)", RFC 4553, DOI 10.17487/RFC4553, June 2006, <http://www.rfc-editor.org/info/rfc4553>.

[RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion Control Algorithms", BCP 133, RFC 5033, DOI 10.17487/RFC5033, August 2007, <http://www.rfc-editor.org/info/rfc5033>.

[RFC5086] Vainshtein, A., Ed., Sasson, I., Metz, E., Frost, T., and P. Pate, "Structure-Aware Time Division Multiplexed (TDM) Circuit Emulation Service over Packet Switched Network (CESoPSN)", RFC 5086, DOI 10.17487/RFC5086, December 2007, <http://www.rfc-editor.org/info/rfc5086>.

[RFC5087] Stein, Y(J)., Shashoua, R., Insler, R., and M. Anavi, "Time Division Multiplexing over IP (TDMoIP)", RFC 5087, DOI 10.17487/RFC5087, December 2007, <http://www.rfc-editor.org/info/rfc5087>.

[RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP Friendly Rate Control (TFRC): Protocol Specification", RFC 5348, DOI 10.17487/RFC5348, September 2008, <http://www.rfc-editor.org/info/rfc5348>.

Appendix A. Loss Probabilities for TDM PWs โœจ

okay so ITU-T Recommendation G.826 [G826] sets limits on the Errored Second Ratio (ESR) and the Severely Errored Second Ratio (SESR) ๐Ÿ’…. To keep things simple, we'll treat an Errored Second (ES) as any second where a TDM bit error happened or a defect indication got detected ๐Ÿซถ. A Severely Errored Second (SES) is an ES where the Bit Error Rate (BER) went over one in one thousand (10^-3) ๐Ÿ˜ญ. Also, if the AIS error condition was detected under the criteria of ITU-T Recommendation G.775 [G775], an SES counts as having happened ๐Ÿ’€. Each ratio is just the ES or SES count divided by the total number of seconds in the measurement interval โœจ.

Every TDM signal runs at 8000 frames per second (higher-rate TDM signals just have longer frames, bestie ๐Ÿ“). So, assuming a whole number of TDM frames per TDM PW packet, the packet rate works out to packets per second = 8000 / (frames per packet). The most common setups are 1, 2, 4, and 8 frames per packet, which give you 8000, 4000, 2000, and 1000 packets per second, respectively ๐ŸŒˆ.

For both E1 and T1 circuits, G.826 allows an ESR of 4% (0.04) and an SESR of 0.2% (0.002). For E3 and T3, the ESR must be no more than 7.5% (0.075), while the SESR stays the same ๐Ÿฆ‹. Let's focus on E1 for a sec. An ESR of 4% works out (in the worst case of isolated, perfectly periodic packet loss) to one packet loss event at most every 25 seconds ๐Ÿงš. Here's the tea though: once a packet is lost, losing another one in that same second doesn't change the ESR, although it might turn the ES into an SES. So for 1, 2, 4, and 8 frames per packet, the maximum allowed packet loss probability is 0.0005%, 0.001%, 0.002%, and 0.004% respectively ๐Ÿ”ฅ.

These super tiny allowed loss probabilities are only for the worst-case scenario, and honestly that's the vibe ๐Ÿ™ˆ. With tail-drop buffers, once packet loss goes above 0.001%, loss bursts are likely to show up. If the lost packets are close enough together (we're skipping the precise details here), then the permitted packet loss ratio goes up by the matching factor, and G.826 doesn't even notice the change ๐Ÿ‘‘. So the worst-case analysis is expected to be extremely pessimistic for real networks. Now let's look at the total opposite extreme, where all packet loss shows up in periodic loss bursts ๐Ÿ’œ. To keep the ESR as low as possible, we'll assume each burst lasts no more than one second, which means we can afford to lose no more than the number of packets sent in one second per burst. As long as these one-second bursts don't take up more than four percent of the time, we're still good on the allowable ESR. So the maximum permissible packet loss ratio is 4%. Iconic, but extremely optimistic, and it also ignores the SESR criteria completely ๐Ÿ˜ญ.

As we said before, an SES gets declared whenever AIS is detected. Structure-aware and structure-agnostic transport are really different here ๐Ÿ’…. When a packet is lost, SAToP outputs an "all-ones" pattern to the TDM circuit, and that gets read as AIS under G.775 [G775]. For E1 circuits, G.775 says AIS is detected when four consecutive TDM frames have no more than 2 alternations. So if a PW packet (or consecutive packets) containing at least four frames is lost, and four or more frames of "all-ones" get output to the TDM circuit, an SES will be declared ๐Ÿ’€. That means burst packet loss, or packets carrying a big pile of TDM frames, make SAToP cause a high SESR, which is 20 times more restricted than the ESR. Meanwhile, structure-aware transport regenerates the correct frame alignment pattern even when the matching packet is gone, so packet loss won't cause an SES to be declared ๐ŸŽ€. This is the main reason SAToP is way more vulnerable to packet loss than the structure-aware methods ๐Ÿซถ.

For realistic networks, the maximum allowed packet loss for SAToP lands somewhere between the super pessimistic estimates and the super optimistic ones โœจ. To get a real number, we modeled the network as a four-state Markov model ๐ŸŒธ. The four states are: a successfully received packet, a packet received within a loss burst, a packet lost within a burst, and a packet lost when not within a burst. This model is an extension of the widely used Gilbert model ๐Ÿ’–. We set the transition probabilities to roughly match anecdotal evidence, which is low background isolated packet loss plus infrequent bursts where most packets are lost ๐Ÿฆ‹. The simulation showed that up to 0.5% average packet loss can happen while the recovered TDM signal still conforms to the G.826 ESR and SESR criteria ๐Ÿ‘‘.

Appendix B. Effect of Packet Loss on Voice Quality for Structure-Aware TDM PWs ๐ŸŽ€

Packet loss in voice traffic makes audio artifacts show up, like choppy, annoying, or even unintelligible speech ๐Ÿ™ˆ. The Voice over IP (VoIP) community has studied the exact effect of packet loss on voice quality in detail, but VoIP results don't directly apply to TDM PWs ๐Ÿ’…. That's because VoIP packets typically carry over 10 milliseconds of the speech signal, while multichannel TDM packets may carry only a single sample, or maybe just a very small number of samples ๐Ÿงš.

The effect of packet loss on TDM PWs has been reported before [PACKET-LOSS] ๐ŸŒˆ. In that study, each packet was assumed to carry a single sample of each TDM timeslot. (Extending this to multiple samples is pretty straightforward and doesn't drastically change the results ๐Ÿ’œ.) Four sample replacement algorithms were compared, and they differ in the value they use to replace the lost sample ๐Ÿ“:

  1. Replacing every lost sample with a preselected constant (e.g., zero or "AIS" insertion).
  2. Replacing a lost sample with the previous sample.
  3. Replacing a lost sample with linear interpolation between the previous and following samples.
  4. Replacing the lost sample with STatistically Enhanced INterpolation (STEIN).

Only the first method works for SAToP transport, since structure awareness is required to identify the individual voice channels ๐Ÿ˜ญ. For structure-aware transport, a lost packet is usually spotted when the next packet shows up, so the following sample is usually available too โœจ. The last algorithm assumes the Linear-Predictive Coding (LPC) speech generation model and figures out lost samples from the available samples both before and after each lost one ๐Ÿฆ‹.

All four algorithms were compared in a controlled experiment ๐Ÿ”ฅ. The speech data came from the English and American English subsets of the ITU-T P.50 Appendix 1 corpus [P50App1], with 16 speakers (eight male and eight female) ๐Ÿซถ. Each speaker said either three or four sentences, for a total of between seven and 15 seconds. The files were filtered to telephony quality using modified IRS filtering and down-sampled to 8 kHz ๐Ÿ’…. Packet loss of 0, 0.25, 0.5, 0.75, 1, 2, 3, 4, and 5 percent was simulated with a uniform random number generator (bursty packet loss was simulated too, but it isn't reported here ๐Ÿ™ˆ). For each file, all four lost-sample replacement methods were applied, and the Mean Opinion Score (MOS) was estimated using PESQ [P862] ๐Ÿ‘‘. Figure 9 shows the PESQ-derived MOS for each of the four replacement methods at packet drop probabilities up to 5% ๐ŸŒธ.

--------------------------------------------------------------------
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I     PESQ-MOS as a function of packet drop probability            I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                     (only in PDF version)                        I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
I                                                                  I
--------------------------------------------------------------------

 Figure 9: PESQ-Derived MOS as a Function of Packet-Drop Probability

For every case, zero insertion gets a lower MOS than the previous sample, which gets a lower MOS than linear interpolation, which is slightly less than statistical interpolation ๐Ÿงš.

Unlike the artifacts that speech compression methods can produce when subjected to buffer loss, packet loss here basically produces additive white impulse noise ๐ŸŒˆ. It sounds like static on an AM radio or crackling on old phonograph records ๐Ÿ’œ. For a given PESQ-derived MOS, this kind of degradation is more acceptable to listeners than the choppiness or tones common in VoIP ๐Ÿ“.

If MOS>4 (full toll quality) is required, then the following packet drop probabilities are allowable:

  • zero insertion - 0.05%
  • previous sample - 0.25%
  • linear interpolation - 0.75%
  • STEIN - 2%

If MOS>3.75 (barely perceptible quality degradation) is acceptable, then the following packet drop probabilities are allowable:

  • zero insertion - 0.1%
  • previous sample - 0.75%
  • linear interpolation - 3%
  • STEIN - 6.5%

If MOS>3.5 (cell phone quality) is tolerable, then the following packet drop probabilities are allowable:

  • zero insertion - 0.4%
  • previous sample - 2%
  • linear interpolation - 8%
  • STEIN - 14%

Authors' Addresses ๐Ÿซถ

Yaakov (Jonathan) Stein
RAD Data Communications
24 Raoul Wallenberg St., Bldg C
Tel Aviv  69719
Israel

Phone: +972 (0)3 645-5389
Email: yaakov_s@rad.com


David L. Black
EMC Corporation
176 South St.
Hopkinton, MA  69719
United States

Phone: +1 (508) 293-7953
Email: david.black@emc.com


Bob Briscoe
BT

Email: ietf@bobbriscoe.net
URI:   http://bobbriscoe.net/
๐Ÿ“ฆ where this page comes from