RFC 5659

An Architecture for Multi-Segment Pseudowire Emulation Edge-to-Edge 🦋

okay bestie, this is RFC 5659 💖 It's an Informational doc from October 2009, so it gives info to the Internet community but isn't an Internet standard. The authors are M. Bocci from Alcatel-Lucent and S. Bryant from Cisco Systems, and it came out of the Network Working Group. Nothing in the header says it obsoletes or updates another RFC, so it doesn't replace or change any older docs 🌸

Abstract ✨

so the big idea is stretching pseudowire emulation across several packet switched network (PSN) segments 🧚 Sometimes every segment of one edge-to-edge emulated service lives on a different provider's PSN. Other times the service starts and ends on the same provider's PSN but still passes through several PSN tunnel segments inside that network. The doc lays out an architectural framework for these multi-segment pseudowires, defines the terminology, and specifies the protocol elements and what each one does 💅 It's giving full blueprint energy.

Status of This Memo 🎀

this memo gives information to the Internet community. It is not an Internet standard of any kind 🙈 Distribution is unlimited, so anyone can read it.

Copyright and License Notice 💜

the copyright is 2009, held by the IETF Trust and the people listed as document authors, all rights reserved 🫶

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were on the publication date. Please actually read those, bestie, because they explain your rights and restrictions for this doc 📜 If you pull Code Components out of this document, they have to include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions. They also come with no warranty, which is the same deal as the BSD License 🌈

Table of Contents 🍓

   1. Introduction ....................................................3
      1.1. Motivation and Context .....................................3
      1.2. Non-Goals of This Document .................................6
      1.3. Terminology ................................................6
   2. Applicability ...................................................8
   3. Protocol Layering Model .........................................8
      3.1. Domain of MS-PW Solutions ..................................9
      3.2. Payload Types ..............................................9
   4. Multi-Segment Pseudowire Reference Model ........................9
      4.1. Intra-Provider Connectivity Architecture ..................11
           4.1.1. Intra-Provider Switching Using ACs .................11
           4.1.2. Intra-Provider Switching Using PWs .................11
      4.2. Inter-Provider Connectivity Architecture ..................11
           4.2.1. Inter-Provider Switching Using ACs .................12
           4.2.2. Inter-Provider Switching Using PWs .................12
   5. PE Reference Model .............................................13
      5.1. Pseudowire Pre-Processing .................................13
           5.1.1. Forwarding .........................................13
           5.1.2. Native Service Processing ..........................14
   6. Protocol Stack Reference Model .................................14
   7. Maintenance Reference Model ....................................15
   8. PW Demultiplexer Layer and PSN Requirements ....................16
      8.1. Multiplexing ..............................................16
      8.2. Fragmentation .............................................17
   9. Control Plane ..................................................17
      9.1. Setup and Placement of MS-PWs .............................17
      9.2. Pseudowire Up/Down Notification ...........................18
      9.3. Misconnection and Payload Type Mismatch ...................18
   10. Management and Monitoring .....................................18
   11. Congestion Considerations .....................................19
   12. Security Considerations .......................................20
   13. Acknowledgments ...............................................23
   14. References ....................................................23
      14.1. Normative References .....................................23
      14.2. Informative References ...................................23

1. Introduction ✨

okay so first we need the basics 🧠 [1] defines the pseudowire architecture, where a pseudowire (PW) both starts and ends at the edges of the same packet switched network (PSN) 💖 The PW label stays the same between the originating and terminating provider edges (PEs). This is what we now call a single-segment pseudowire (SS-PW) 🦋

this doc builds on [1] so pseudowires can be extended through multiple PSN tunnels, which makes them point-to-point across several hops 🌸 Those are the multi-segment pseudowires (MS-PWs) 👑 The use cases for MS-PWs and the requirements that come with them are covered in [5] 🫶

1.1. Motivation and Context 💖

Okay bestie, so RFC 3985 covers the case where a PW spans a single segment between two PEs. These are called single-segment pseudowires (SS-PWs), and they give you point-to-point connectivity between two edges of a provider network. 🦋 But now there's a need to build multi-segment pseudowires too. Those needs are laid out in RFC 5254 [5], and they tackle three main problems:

i. How to constrain the density of the mesh of PSN tunnels when the number of PEs grows to many hundreds or thousands, while minimizing the complexity of the PEs and P-routers. 🌸

ii. How to provide PWs across multiple PSN routing domains or areas in the same provider. ✨

iii. How to provide PWs across multiple provider domains and different PSN types. 👑

Let's look at a single PW domain, like the one in Figure 1. There are 4 PEs, and PWs have to be provided from any PE to any other PE. 🧚 You can support this by setting up a full mesh of PSN tunnels between the PEs, which needs a full mesh of LDP signaling adjacencies between the PEs too. So any PE can reach any other PE through one direct PSN tunnel that only intermediate P-routers switch (they aren't shown in the figure). In that setup, each PW is an SS-PW. A PE has to terminate all the pseudowires carried on the PSN tunnels that end on it, following the architecture of RFC 3985. This works fine for a small number of PEs, but the number of PEs, PSN tunnels, and signaling adjacencies grows in proportion to the square of the number of PEs. 💀

                                   ,,..--..,,_
                               .-``           `'.,
                       +-----+`                   '+-----+
                       | PE1 |---------------------| PE2 |
                       |     |---------------------|     |
                       +-----+      PSN Tunnel     +-----+
                       / ||                          || \
                      /  ||                          ||  \
                     |   ||                          ||   |
                     |   ||         PSN              ||   |
                     |   ||                          ||   |
                      \  ||                          ||  /
                       \ ||                          || /
                        \||                          ||/
                       +-----+                     +-----+
                       | PE3 |---------------------| PE4 |
                       |     |---------------------|     |
                       +-----+`'.,_           ,.'` +-----+
                                   `'''---''``

Figure 1: PWs Spanning a Single PSN with Partial Mesh of PSN Tunnels

For economy's sake, the edge PEs that terminate the attachment circuits (ACs) are often tiny devices built to a very low cost, with limited processing power. 🎀 Imagine one particular PE sitting at the edge of a provider network, terminating N PWs to and from N different remote PEs. That means it needs N PW signaling adjacencies set up and maintained. 😭 But if the edge PE attaches to a single intermediate PE that's able to switch the PW, then the edge PE only needs one adjacency to signal and maintain all N PWs. The intermediate switching PE is the bigger device, and it needs M signaling adjacencies. Statistically, that's less than tN, where t is the number of edge PEs it's serving. It's giving efficiency! Similarly, if the PWs run over TE PSN tunnels, there's a statistical drop in the number of TE PSN tunnels that have to be set up and maintained between the various PEs. 🔥

So a solution that's more efficient for large numbers of PEs, especially for the control plane, is to support a partial mesh of PSN tunnels between the PEs, as shown in Figure 1. 💅 For example, take a PW service whose endpoints are PE1 and PE4. The pseudowires for this can take the path PE1->PE2->PE4 and, instead of terminating at PE2, get switched between ingress and egress PSN tunnels on that PE. This needs a capability in PE2 that can concatenate PW segments PE1-PE2 to PW segments PE2-PE4. The end-to-end PW is known as a multi-segment PW. 🌈

Figure 1 shows a simple, flat PSN topology. But big provider networks usually aren't flat. They're made of many domains connected together to provide edge-to-edge services. Each element in each domain has a specialized role, for example, supporting different PSN types or using different routing protocols. 🍓

Here's an example application, shown in Figure 2. The provider's network is split into three domains: two access domains and the core domain. The access domains are the edge of the provider's network, where services get delivered. In the access domain, simplicity is needed so the cost of the network stays low. ✨ The core domain has to support all of the aggregated services from the access domains, and its design goals are scalability, performance, and information hiding (i.e., minimal state). The core must not be exposed to the state associated with large numbers of individual edge-to-edge flows. In other words, the core has to be simple and fast. 🦋

            <-------- Edge to Edge Emulated Services ------->

                ,'    .      ,-`       `',       ,'    .
               /       \   .`             `,    /       \
              /        \  /                 ,  /        \
       AC  +----+     +----+               +----+       +----+    AC
        ---| PE |-----| PE |---------------| PE |-------| PE |---
           |  1 |     |  2 |               | 3  |       | 4  |
           +----+     +----+               +----+       +----+
              \        /  \                 /  \        /
               \       /  \      Core       `   \       /
                `,    `     .             ,`     `,    `
                  '-'`       `.,       _.`         '-'`
               Access 1         `''-''`         Access 2

Figure 2: Multi-Domain Network Model

In a traditional layer 2 network, the interconnection points between domains are where services in the access domains get aggregated for transport across the core to other access domains. 🌸 In an IP network, those interconnection points could also be interworking points between different types of IP networks, e.g., those with MPLS and those without, and points where network policies can be applied. 💜

The same model can also apply to inter-provider services, where a single PW spans a number of separate provider networks to connect ACs living on PEs in different provider networks. 🫶 Here, each provider will typically keep their own PE at the border of their network so they can apply policies such as security and Quality of Service (QoS) to PWs entering their network. So the connection between the domains will normally be a link between two PEs on the border of each provider's network. 👑

Now let's think about applying this model to PWs. PWs use tunneling mechanisms such as MPLS so the underlying PSN can emulate the characteristics of the native service. ⭐ One solution to the multi-domain model above is to extend PSN tunnels edge-to-edge between all of the PEs in access domain 1 and all of the PEs in access domain 2. But that needs a large number of PSN tunnels, as described above, and it also exposes the access and core of the network to undesirable complexity. 🙈 The alternative is to constrain the complexity to the network domain interconnection points (PE2 and PE3 in the example above). Pseudowires between PE1 and PE4 would then be switched between PSN tunnels at those interconnection points, so PWs from many PEs in the access domains can be aggregated across just a few PSN tunnels in the core. The PEs in the access domains only need to maintain direct signaling sessions and PSN tunnels with other PEs in their own domain, which keeps the complexity of the access domains minimal. 🧚

1.2. Non-Goals of This Document 🙅‍♀️

The following are non-goals for this document:

• The on-the-wire specification of PW encapsulations.
• The detailed specification of mechanisms for establishing and maintaining multi-segment pseudowires.

1.3. Terminology 📚

The terminology specified in RFC 3985 [1] and RFC 4026 [2] applies. On top of that, we define the following terms:

• PW Terminating Provider Edge (T-PE). A PE where the customer-facing attachment circuits (ACs) are bound to a PW forwarder. A terminating PE is present in the first and last segments of an MS-PW. This incorporates the functionality of a PE as defined in RFC 3985.
• Single-Segment Pseudowire (SS-PW). A PW set up directly between two T-PE devices. The PW label is unchanged between the originating and terminating T-PEs.
• Multi-Segment Pseudowire (MS-PW). A static or dynamically configured set of two or more contiguous PW segments that behave and function as a single point-to-point PW. Each end of an MS-PW, by definition, terminates on a T-PE.
• PW Segment. A part of a single-segment or multi-segment PW, which traverses one PSN tunnel in each direction between two PE devices, T-PEs, and/or S-PEs (switching PE).
• PW Switching Provider Edge (S-PE). A PE capable of switching the control and data planes of the preceding and succeeding PW segments in an MS-PW. The S-PE terminates the PSN tunnels of the preceding and succeeding segments of the MS-PW. It therefore includes a PW switching point for an MS-PW. A PW switching point is never the S-PE and the T-PE for the same MS-PW. A PW switching point runs necessary protocols to set up and manage PW segments with other PW switching points and terminating PEs. An S-PE can exist anywhere a PW must be processed or policy applied. It is therefore not limited to the edge of a provider network.

Note that it was originally anticipated that S-PEs would only be deployed at the edge of a provider network, where they would be used to switch the PWs of different service providers. However, as the design of MS-PW progressed, other applications for MS-PW were recognized. By this time, S-PE had become the accepted term for the equipment, even though they were no longer universally deployed at the provider edge.

• PW Switching. The process of switching the control and data planes of the preceding and succeeding PW segments in a MS-PW.
• PW Switching Point. The reference point in an S-PE where the switching takes place, e.g., where PW label swap is executed.
• Eligible S-PE or T-PE. An eligible S-PE or T-PE is a PE that meets the security and privacy requirements of the MS-PW, according to the network operator's policy.
• Trusted S-PE or T-PE. A trusted S-PE or T-PE is a PE that is understood to be eligible by its next-hop S-PE or T-PE, while a trust relationship exists between two S-PEs or T-PEs if they mutually consider each other to be eligible. 🫶

2. Applicability ✨

okay so an MS-PW is basically ONE single PW that, for technical or administrative reasons, gets chopped up into a bunch of concatenated hops 🔗. From the perspective of a Layer 2 Virtual Private Network (L2VPN), an MS-PW is literally indistinguishable from an SS-PW 💅. So these two setups are equivalent from the T-PE's point of view:

    +----+                                                  +----+
    |TPE1+--------------------------------------------------+TPE2|
    +----+                                                  +----+

    |<---------------------------PW----------------------------->|

    +----+              +---+           +---+               +----+
    |TPE1+--------------+SPE+-----------+SPE+---------------+TPE2|
    +----+              +---+           +---+               +----+

                       Figure 3: MS-PW Equivalence

Now, an MS-PW might need stuff like node discovery and path signaling to get built, but it should not be confused with an L2VPN system, which needs those services too 🙃. A Virtual Private Wire Service (VPWS) connects its endpoints via a set of PWs. MS-PW is the mechanism that abstracts building those complex PWs away from building the L2VPN itself 🧚. So a T-PE can be a super simple edge device, and an S-PE can be an aggregation device that absorbs all the complexity of continuing the PW across the core of one or more service provider networks to another T-PE sitting at the edge of the network 🫶.

Also, MS-PW isn't just for traditional L2VPNs. It's applicable to providing connectivity across a transport network based on packet switching technology, e.g., the MPLS Transport Profile (MPLS-TP) [6], [8] 🌈. Those networks use pseudowires to support the transport and aggregation of all services, and they need deterministic characteristics and behavior from the network 😭. The operational requirements of these networks may need pseudowire segments that can be established and maintained in the absence of a control plane, and they may also need the operational independence of PW maintenance from the underlying PSN 🦋.

3. Protocol Layering Model

the protocol layering model specified in RFC 3985 applies to MS-PWs, with one clarification 🎀: the pseudowires may be considered to be a separate layer to the PSN tunnel. Like, although a PW segment follows the path of the PSN tunnel between S-PEs, the MS-PW is independent of the PSN tunnel's routing, operations, signaling, and maintenance 💅. The design of PW routing domains should not imply that the underlying PSN routing domains are the same. BUT MS-PWs will reuse the protocols of the PSN and may, if applicable, use information extracted from the PSN, e.g., reachability 🤝.

3.1. Domain of MS-PW Solutions

PWs provide the Encapsulation Layer (the method of carrying various payload types) and the interface to the PW Demultiplexer Layer 🍓. Other layers provide the following:

• PSN tunnel setup, maintenance, and routing
• T-PE discovery

Not all PEs may be capable of providing S-PE functionality, so not every PE is a sure thing 🙈. Connectivity to the next-hop S-PE or T-PE must be provided by a PSN tunnel, according to [1]. The selection of which set of S-PEs to use to reach a given T-PE is considered to be within the scope of MS-PW solutions ✨.

3.2. Payload Types

MS-PWs are applicable to all PW payload types, no exceptions 👑. Encapsulations defined for SS-PWs are also used for MS-PW without change. Where the PSN types for each segment of an MS-PW are identical, the PW types of each segment must also be identical 🔥. However, if different segments run over different PSN types, the encapsulation may change, but the PW segments must be of an equivalent PW type. In other words, the S-PE must not need to process the PW payload to provide translation 🫢.

4. Multi-Segment Pseudowire Reference Model 🎀

the pseudowire emulation edge-to-edge (PWE3) reference architecture for the single-segment case is shown in [1] 📚. This architecture applies to the case where a PSN tunnel extends between two edges of a single PSN domain to transport a PW with endpoints at those edges.

    Native  |<------Multi-Segment Pseudowire------>|  Native
    Service |         PSN              PSN         |  Service
     (AC)   |     |<-Tunnel->|     |<-Tunnel->|    |   (AC)
       |    V     V     1    V     V    2     V    V     |
       |    +----+           +-----+          +----+     |
+----+ |    |TPE1|===========|SPE1 |==========|TPE2|     | +----+
|    |------|..... PW.Seg't1....X....PW.Seg't3.....|-------|    |
| CE1| |    |    |           |     |          |    |     | |CE2 |
|    |------|..... PW.Seg't2....X....PW.Seg't4.....|-------|    |
+----+ |    |    |===========|     |==========|    |     | +----+
     ^      +----+           +-----+          +----+       ^
     |   Provider Edge 1        ^        Provider Edge 2   |
     |                          |                          |
     |                          |                          |
     |                  PW switching point                 |
     |                                                     |
     |<------------------ Emulated Service --------------->|

                  Figure 4: MS-PW Reference Model

Figure 4 stretches that architecture out to show a multi-segment case 🦋. The PEs that provide services to CE1 and CE2 are Terminating PE1 (T-PE1) and Terminating PE2 (T-PE2), respectively. A PSN tunnel extends from T-PE1 to Switching PE1 (S-PE1) across PSN1, and a second PSN tunnel extends from S-PE1 to T-PE2 across PSN2. PWs are used to connect the attachment circuits (ACs) attached to PE1 to the corresponding ACs attached to T-PE2 🌸.

Each PW segment on the tunnel across PSN1 is switched to a PW segment in the tunnel across PSN2 at S-PE1, which completes the multi-segment PW (MS-PW) between T-PE1 and T-PE2 💖. So S-PE1 is the PW switching point. PW segment 1 and PW segment 3 are segments of the same MS-PW, while PW segment 2 and PW segment 4 are segments of another MS-PW. PW segments of the same MS-PW (e.g., PW segment 1 and PW segment 3) must be of equivalent PW types, as described in Section 3.2, while PSN tunnels (e.g., PSN1 and PSN2) may be of the same or different PSN types 🔁. An S-PE switches an MS-PW from one segment to another based on the PW demultiplexer, i.e., a PW label that may take one of the forms defined in Section 5.4.1 of RFC 3985 [1] 🏷️.

Note that although Figure 4 only shows a single S-PE, a PW may transit more than one S-PE along its path 👀. This architecture is applicable when the S-PEs are statically chosen, or when they are chosen using a dynamic path-selection mechanism. Both directions of an MS-PW must traverse the same set of S-PEs on a reciprocal path. Note that although the S-PE path is therefore reciprocal, the path taken by the PSN tunnels between the T-PEs and S-PEs might not be reciprocal, due to choices made by the PSN routing protocol 🤷‍♀️.

4.1. Intra-Provider Connectivity Architecture

There is a requirement to deploy PWs edge-to-edge in large service provider networks (RFC 5254 [5]) 🏙️. Such networks typically encompass hundreds or thousands of aggregation devices at the edge, each of which would be a PE. These networks may be partitioned into separate metro and core PW domains, where the PEs are interconnected by a sparse mesh of tunnels.

Whether or not the network is partitioned into separate PW domains, there is also a requirement to support a partial mesh of traffic-engineered PSN tunnels 🧠.

The architecture shown in Figure 4 can be used to support such cases ✨. PSN1 and PSN2 may be in different administrative domains or access regions, core regions, or metro regions within the same provider's network. PSN1 and PSN2 may also be of different types. For example, S-PEs may be used to connect PW segments traversing metro networks of one technology, e.g., statically allocated labels, with segments traversing an MPLS core network 🎢.

Alternatively, T-PE1, S-PE1, and T-PE2 may reside at the edges of the same PSN 🫶.

4.1.1. Intra-Provider Switching Using ACs

In this model, the PW reverts to the native service AC at the domain boundary PE. This AC is then connected to a separate PW on the same PE. In this case, the reference models of RFC 3985 apply to each segment and to the PEs. The remaining PE architectural considerations in this document do not apply to this case 🙅‍♀️.

4.1.2. Intra-Provider Switching Using PWs

In this model, PW segments are switched between PSN tunnels that span portions of a provider's network, without reverting to the native service at the boundary 🔄. For example, in Figure 4, PSN1 and PSN2 would be portions of the same provider's network 💜.

4.2. Inter-Provider Connectivity Architecture

Inter-provider PWs may need to be switched between PSN tunnels at the provider boundary, in order to minimize the number of tunnels required to provide PW-based services to CEs attached to each provider's network 🧁. On top of that, the following may need to be implemented on a per-PW basis at the provider boundary:

• Operations, Administration, and Maintenance (OAM). Note that this is synonymous with 'Operations and Maintenance' referred to in RFC 5254 [5].
• Authentication, Authorization, and Accounting (AAA)
• Security mechanisms

Further security-related architectural considerations are described in Section 12 🔐.

4.2.1. Inter-Provider Switching Using ACs

In this model, the PW reverts to the native service at the provider boundary PE. This AC is then connected to a separate PW at the peer provider boundary PE. In this case, the reference models of RFC 3985 apply to each segment and to the PEs. This is similar to the case in Section 4.1.1, except that additional security and policy enforcement measures will be required 🛡️. The remaining PE architectural considerations in this document do not apply to this case.

4.2.2. Inter-Provider Switching Using PWs

In this model, PW segments are switched between PSN tunnels in each provider's network, without reverting to the native service at the boundary 🌍. This architecture is shown in Figure 5. Here, S-PE1 and S-PE2 are provider border routers. PW segment 1 is switched to PW segment 2 at S-PE1. PW segment 2 is then carried across an inter-provider PSN tunnel to S-PE2, where it is switched to PW segment 3 in PSN2 🥹.

          |<------Multi-Segment Pseudowire------>|
          |       Provider         Provider      |
     AC   |    |<----1---->|     |<----2--->|    |  AC
      |   V    V           V     V          V    V  |
      |   +----+     +-----+     +----+     +----+  |
+----+   |   |    |=====|     |=====|    |=====|    |  |    +----+
|    |-------|......PW.....X....PW.....X...PW.......|-------|    |
| CE1|   |   |    |Seg 1|     |Seg 2|    |Seg 3|    |  |    |CE2 |
+----+   |   |    |=====|     |=====|    |=====|    |  |    +----+
     ^       +----+     +-----+     +----+     +----+       ^
     |       T-PE1       S-PE1       S-PE2     T-PE2        |
     |                     ^          ^                     |
     |                     |          |                     |
     |                  PW switching points                 |
     |                                                      |
     |                                                      |
     |<------------------- Emulated Service --------------->|

              Figure 5: Inter-Provider Reference Model

5. PE Reference Model 🌟

5.1. Pseudowire Pre-Processing 🎀

Okay so pseudowire pre-processing happens at the T-PEs, and RFC 3985 is where that's all spelled out 📖 The S-PEs have their own processing rules, and those are in the sections below ✨

5.1.1. Forwarding 🦋

Each forwarder in the S-PE takes packets from one PW segment on the ingress PSN-facing interface and forwards them to one PW segment on the egress PSN-facing interface. It's giving pass-through energy 💅

The forwarder picks the egress segment PW based on the ingress PW label. The mapping from ingress to egress PW label can be set up statically or dynamically, so you get to choose what works best 🌸 Figure 6 shows how a single forwarder is linked to each PW segment at the S-PE:

           +------------------------------------------+
           |                S-PE Device               |
           +------------------------------------------+
 Ingress   |             |             |              |   Egress
PW instance |   Single    |             |    Single    | PW Instance
<==========>X PW Instance +  Forwarder  + PW Instance  X<==========>
           |             |             |              |
           +------------------------------------------+

                 Figure 6: Point-to-Point Service

Other PW-to-forwarder mappings are still for further study 🔮

5.1.2. Native Service Processing 🫶

There is no native service processing in the S-PEs. Zero. None. The S-PEs just keep things moving 💖

6. Protocol Stack Reference Model 🧚

Okay this is the fun one 🌈 Figure 7 shows the protocol stack reference model for multi-segment PWs:

+-----------+                                  +-----------+
|  Emulated |                                  |  Emulated |
|  Service  |                                  |  Service  |
|(e.g., ATM)|<======= Emulated Service =======>|(e.g., ATM)|
+-----------+                                  +-----------+
| Payload   |                                  | Payload   |
|  Encap.   |<=== Multi-segment Pseudowire ===>|  Encap.   |
+-----------+            +--------+            +-----------+
| PW Demux  |<PW Segment>|PW Demux|<PW Segment>| PW Demux  |
+-----------+            +--------+            +-----------+
|PSN Tunnel,|<PSN Tunnel>|  PSN   |<PSN Tunnel>|PSN Tunnel,|
| PSN & PHY |            |Physical|            | PSN & PHY |
| Layers    |            | Layers |            |  Layers   |
+----+------+            +--------+            +-----+-----+
     |            ..........   |   ..........        |
     |           /          \  |  /          \       |
     +==========/    PSN     \===/    PSN     \======+
                \  domain 1  /   \  domain 2  /
                 \__________/     \__________/

Figure 7: Multi-Segment PW Protocol Stack

So the MS-PW gives the CE an emulated physical or virtual connection to its peer at the far end 🦋 Native service PDUs coming from the CE get wrapped in an Encapsulation Layer, and the sending T-PE adds a PW demultiplexer. The PDU then travels across the PSN domain over the PSN transport tunnel 🚀 At the receiving S-PE, the existing PW demultiplexer gets swapped out for the demultiplexer of the next segment, and the PDU goes out over the transport tunnel in PSN2.

If the ingress and egress PSN domains of the S-PE are the same type, say both are MPLS PSNs, it's just a simple label swap operation, as described in Section 3.13 of RFC 3031 [3] 🔁 But if the ingress and egress PSNs are different types, like MPLS and L2TPv3, the ingress PW demultiplexer gets removed (popped), then the S-PE maps it to the egress PW demultiplexer and inserts (pushes) that one instead 💅

Policies can also be applied to the PW at this point 👑 Examples include admission control, rate control, QoS mappings, and security. Then the receiving T-PE removes the PW demultiplexer and restores the payload to its native format so it can go to the destination CE 🎀

> Where the encapsulation format is different, like MPLS and L2TPv3, the payload encapsulation may be translated at the S-PE. 🌸

## 7. Maintenance Reference Model 🔧

Figure 8 shows the maintenance reference model for multi-segment pseudowires 🧐

V +----+ +-----+ +----+ V +----+ |TPE1|===========|SPE1 |===========|TPE2| +----+

+----+ | |===========| |===========| | +----+ ^ +----+ +-----+ +----+ ^

Figure 8: MS-PW Maintenance Reference Model

RFC 3985 covers CE (end-to-end) and PSN tunnel signaling, plus PW/PE maintenance 📖 CE and PSN tunnel signaling work the way RFC 3985 says. But with MS-PWs, the signaling between PEs has two layers going on at once: an edge-to-edge context and a hop-by-hop context 🤯 Signaling and maintenance between T-PEs and S-PEs, and between adjacent S-PEs, is what sets up, maintains, and tears down the MS-PW segments. It also handles coordinating the parameters for each switching point as well as for the MS-PW endpoints. Iconic multitasking 💜

## 8. PW Demultiplexer Layer and PSN Requirements 💫

### 8.1. Multiplexing 🌷

The PW Demultiplexer Layer at the S-PE has one job: demultiplex PWs from ingress PSN tunnels, then multiplex them into egress PSN tunnels 🔀 Each PW can hold multiple native service circuits, like multiple ATM virtual circuits (VCs). But the S-PEs can't see that level of multiplexing, so they don't change it, because they don't have a Native Service Processor (NSP) 🙈

### 8.2. Fragmentation 🧩

If fragmentation is going to be used in an MS-PW, T-PEs and S-PEs have to make sure that fragmented PW payloads can be correctly reassembled for delivery to the destination attachment circuit 🫠

An S-PE is not required to make any attempt to reassemble a fragmented PW payload. It can still choose to, though, if it knows, for example, that a downstream PW segment doesn't support reassembly 🤷‍♀️

An S-PE may fragment a PW payload using [4] ✨

## 9. Control Plane 🎛️

### 9.1. Setup and Placement of MS-PWs 🗺️

For multi-segment pseudowires, the intermediate PW switching points can be statically provisioned or chosen dynamically 🌈

For the static case, there are two options for exchanging the PW labels:

- By configuration at the T-PEs or S-PEs.
- By signaling across each segment using a dynamic maintenance protocol.

So a multi-segment pseudowire can be a mix: some segments have statically configured labels and others have signaled labels 🎀

For dynamic choice of the PW switching points, there are two options for selecting the path of the MS-PW:

- T-PEs determine the full path of the PW through intermediate switching points. This may be either static or based on a dynamic PW path-selection mechanism.
- Each T-PE and S-PE makes a local decision about which next-hop S-PE to choose to reach the target T-PE. This choice is made either using locally configured information or by using a dynamic PW path-selection mechanism.

### 9.2. Pseudowire Up/Down Notification 🟢

A multi-segment PW is a bunch of concatenated PW segments chained together, so the emulated service only counts as up when every one of those PW segments and PSN tunnels is working and operational along the entire path of the MS-PW 🔗

If a native service needs bi-directional connectivity, the emulated service can only be signaled as up when the PW segments and PSN tunnels (if used) are functional and operational in both directions ↔️

RFC 3985 describes the architecture of failure and other status notification mechanisms for PWs. Those mechanisms are needed in multi-segment pseudowires too 📣 Also, if a failure notification mechanism is provided for consecutive segments of the same PW, the S-PE has to pass those notifications along between the consecutive concatenated segments 🏃‍♀️

### 9.3. Misconnection and Payload Type Mismatch 🚨

Misconnection and payload type mismatch can both happen with PWs 😬 Misconnection can breach the integrity of the system, and payload mismatch can disrupt the customer network. Either way, there are security and operational concerns, so this is serious stuff 💀

The services of the underlying tunneling mechanism or the PW control and OAM protocols can be used to make sure the identity of the PW next hop is what you expect 🔍 As part of PW setup, a PW-TYPE identifier gets exchanged. The forwarder and the NSP at the T-PEs use it to check that the ACs are compatible. S-PEs can use it too, to make sure concatenated segments of a given MS-PW are compatible, or that an MS-PW isn't misconnected into a local AC. It's also possible to do an end-to-end connection verification to check the integrity of the PW, verify the identity of S-PEs, check the correct connectivity at S-PEs, and verify the identity of the T-PE 🫶

## 10. Management and Monitoring 📊

The management and monitoring described in RFC 3985 applies here 📖

The MS-PW architecture adds some extra things to think about for management and monitoring, and those need to be reflected in the design of maintenance tools and in additional management objects for MS-PWs 🛠️

First up: each S-PE is a new spot where defects can show up along the path of the PW 🐛 To troubleshoot MS-PWs, management and monitoring should be able to work on a subset of the segments of an MS-PW, not just edge-to-edge. So connectivity verification mechanisms should be able to troubleshoot and tell apart the connectivity between T-PEs and intermediate S-PEs, as well as the connectivity between T-PE and T-PE 🔎

Second: the set of S-PEs and P-routers along the MS-PW path may be less optimal than a path between the T-PEs picked only by the underlying PSN routing protocols 🛤️ That's because the S-PEs get chosen by the MS-PW path selection mechanism, not by the PSN routing protocols. So troubleshooting mechanisms should be provided to verify the set of S-PEs that an MS-PW traverses to reach a T-PE 🕵️‍♀️

Some of the S-PEs and T-PEs for an MS-PW may live in a different service provider's PSN domain from the operator who started setting up the MS-PW 🌍 Those situations may call for remote management of the MS-PW, which can securely operate across provider boundaries 🔐

# 11. Congestion Considerations 🚦

okay so these congestion considerations apply to MS-PWs, bestie 💖 They're *on top of* the stuff in RFC 3985 [1], [7], and whatever RFC covers each specific PW type.

The control plane and the data plane fate-share in traditional IP networks 🫶 Translation: if the data plane gets congested, the control plane can get degraded too. Like, when the network is chill and running within its design limits, it's expected that the network is designed to avoid this drama. BUT (and this is the iconic part) MS-PW mechanisms should also think about what happens when congestion *does* hit, like when the network gets stretched past its design limits during something unexpected like a network failure 😭

Okay so, congestion inside one provider's network can be handled by engineering the network properly, so the PW traffic can never cause congestion in the underlying PSN 💅 However, a big number of MS-PWs are expected to be deployed for inter-provider services, and that's where it gets messy. A provider that starts setting up an MS-PW at a T-PE may have *no way* of guaranteeing it won't cause congestion in a downstream PSN 💀 A specific PSN can protect itself from excess PW traffic by policing all PWs at the S-PE at the provider border. Not gonna lie though, that may not be effective when the PSN tunnel across a provider uses the transit services of another provider that can't tell PW traffic apart from ordinary, TCP-controlled IP traffic 🙈

So basically, each segment of an MS-PW needs to implement congestion detection and congestion control mechanisms wherever it isn't possible to explicitly provision enough capacity to avoid congestion 🌸

In a lot of cases, only the T-PEs have enough info about each PW to fairly apply congestion control 👑 So T-PEs need to be aware of which of their PWs are causing congestion in a downstream PSN, and also know their native service characteristics, so they can apply congestion control accordingly ✨ That means S-PEs need to propagate PSN congestion state information between their downstream and upstream directions 🫶 If the MS-PW transits many S-PEs, it may take some time for that congestion state info to travel from the congested PSN segment all the way back to the source T-PE, which delays when congestion control kicks in 😩 Congestion control in the S-PE at the border of the congested PSN can enable a faster response and potentially reduce how long the congestion lasts. Main character energy 🦋

Beyond protecting the underlying PSN, consistent QoS and traffic engineering mechanisms should be used on each segment of an MS-PW to support the requirements of the emulated service 🎀 The QoS treatment a PW packet gets at an S-PE may come from context information about the PW (for example, traffic or QoS parameters signaled to the S-PE by an MS-PW control protocol), or from PSN-specific QoS flags in the PSN tunnel label or PW demultiplexer. Those flags look like: TC bits in either the label switched path (LSP) or PW label for an MPLS PSN, or the DS field of the outer IP header for L2TPv3 🧚

# 12. Security Considerations 🔐

Okay, security time, and I'm dead serious about this one 💀 The security considerations described in RFC 3985 [1] apply here too. Detailed security requirements for MS-PWs are specified in RFC 5254 [5]. This section describes the architectural implications of those requirements 🔥

The security implications for T-PEs are similar to those for PEs in single-segment pseudowires. HOWEVER, S-PEs are a point in the network where the PW label is exposed to additional processing, which is kinda sus 👀 An S-PE or T-PE must trust that the context of the MS-PW is maintained by a downstream S-PE. OAM tools must be able to verify the identity of the far end T-PE to the satisfaction of the network operator. Additional consideration needs to be given to the security of the S-PEs, both at the data plane and the control plane, especially when they're dynamically selected and/or when the MS-PW transits the networks of multiple operators 🚨

There's an implicit trust relationship between the initiator of an MS-PW, the T-PEs, and the S-PEs along the MS-PW's path 🫶 Basically, the T-PE trusts the S-PEs to process and switch PWs without compromising the security or privacy of the PW service. An S-PE should not select a next-hop S-PE or T-PE unless it knows that node would be considered eligible (as defined in Section 1.3) by the originator of the MS-PW. For dynamically placed MS-PWs, this can be achieved by letting the T-PE explicitly specify the path of the MS-PW 🗺️ When the MS-PW is dynamically created using a signaling protocol, an S-PE or T-PE should determine the authenticity of the peer entity it receives the request from, and whether that request complies with policy 💅

Where an MS-PW crosses a border between one provider and another, the MS-PW segment endpoints (S-PEs or T-PEs) or, for the PSN tunnel, P-routers typically live on the same nodes as the Autonomous System Border Routers (ASBRs) that interconnect the two providers. Either way, an S-PE in one provider is connected to a limited number of trusted T-PEs or S-PEs in the other provider 🤝 That number is bounded and isn't expected to create a scaling issue for the control plane authentication mechanisms, which is a relief honestly 😌

Directly interconnecting the S-PEs/T-PEs using a physically secure link, and enabling signaling and routing authentication between them, gets rid of the possibility of receiving an MS-PW signaling message or packet from an untrusted peer 🔒 The S-PEs/T-PEs are the security policy enforcement points for the MS-PW, while the ASBRs are the security policy enforcement points for the provider's PSNs. Here's the architecture, bestie 👇

AC | |<----1---->| |<----2--->| | AC

+---+ | | |=====| |=====| |=====| | | +---+

+---+ | | |=====| |=====| |=====| | | +---+ ^ +----+ +-----+ ^ +----+ +----+ ^

Figure 9: Directly Connected Inter-Provider Reference Model

Alternatively, the P-routers for the PSN tunnel may live on the ASBRs, while the S-PEs or T-PEs sit behind the ASBRs within each provider's network. A limited number of trusted inter-provider PSN tunnels interconnect the provider networks 🌈 Here's how that looks:

AC | |<------1----->| |<-----2------->| | AC

+---+ | | |=====| |===============| |=====| | | +---+

+---+ | | |=====| |===============| |=====| | | +---+ ^ +---+ +---+ +--+ ^ +--+ +---+ +---+ ^

Figure 10: Indirectly Connected Inter-Provider Reference Model

Quality of Service requests need *special* attention 🎀 Inappropriate use of priority can hurt the service guarantees given to other PWs, so be careful! Consideration also needs to be given to avoiding spoofing of the PW demultiplexer 🕵️

Where an S-PE provides interconnection between different providers, security considerations similar to those for ASBRs apply. In particular, peer entity authentication should be used ✨

Where an S-PE also supports T-PE functionality, mechanisms should be provided to make sure MS-PWs are switched correctly to the appropriate outgoing PW segment, rather than to a local AC. Other mechanisms for PW endpoint verification may also be used to confirm the correct PW connection before the attachment circuits get enabled 🫶

# 13. Acknowledgments 💜

The authors gratefully acknowledge the input of Mustapha Aissaoui, Dimitri Papadimitrou, Sasha Vainshtein, and Luca Martini. Thank you, queens 👑

# 14. References 📚

## 14.1. Normative References

[1] Bryant, S., Ed., and P. Pate, Ed., "Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture", RFC 3985, March 2005.

[2] Andersson, L. and T. Madsen, "Provider Provisioned Virtual Private Network (VPN) Terminology", RFC 4026, March 2005.

[3] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, January 2001.

[4] Malis, A. and M. Townsley, "Pseudowire Emulation Edge-to-Edge (PWE3) Fragmentation and Reassembly", RFC 4623, August 2006.

## 14.2. Informative References

[5] Bitar, N., Ed., Bocci, M., Ed., and L. Martini, Ed., "Requirements for Multi-Segment Pseudowire Emulation Edge-to-Edge (PWE3)", RFC 5254, October 2008.

[6] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., Sprecher, N., and S. Ueno, "Requirements of an MPLS Transport Profile", RFC 5654, September 2009.

[7] Bryant, S., Davie, B., Martini, L., and E. Rosen, "Pseudowire Congestion Control Framework", Work in Progress, June 2009.

[8] Bocci, M., Bryant, S., and L. Levrau, "A Framework for MPLS in Transport Networks", Work in Progress, August 2009.

# Authors' Addresses 🦋

Matthew Bocci Alcatel-Lucent Voyager Place, Shoppenhangers Road, Maidenhead, Berks, UK Phone: +44 1633 413600 EMail: matthew.bocci@alcatel-lucent.com

Stewart Bryant Cisco Systems 250, Longwater, Green Park, Reading, RG2 6GB, United Kingdom EMail: stbryant@cisco.com

---

<rainbow>That's the whole RFC, bestie!! 🌈✨💅</rainbow>