RFC 6383

Advice on When It Is Safe to Start Sending Data on Label Switched Paths Established Using RSVP-TE

📅 published September 2011, from ietf

RFC 6383 💖 Advice on When It Is Safe to Start Sending Data on Label Switched Paths Established Using RSVP-TE

  • Number: RFC 6383
  • Authors: K. Shiomoto (NTT) and A. Farrel (Old Dog Consulting) 🎀
  • Category: Informational
  • ISSN: 2070-1721
  • Date: September 2011
  • Obsoletes / Updates: neither is listed in this document's header ✨

Abstract 🌸

Okay so RSVP got a major glow-up 💅 to do Traffic Engineering (TE) in MPLS and GMPLS networks. It lets signaling exchanges set up Label Switched Paths (LSPs), which are basically end-to-end data paths that run through nodes and links. As the signaling messages get processed, every node gets programmed with "cross-connect" info that tells it how to forward the data it receives.

End points of an LSP need to know when it's safe to start sending data, so the data doesn't get misdelivered, and so the safety issues specific to optical data-plane tech are handled too 🔥. Meanwhile, every label switching router along the LSP's path needs to know when to program its data plane compared to when it sends and receives control-plane messages 🧚.

This doc clarifies and summarizes the RSVP-TE protocol exchanges related to programming cross-connects along an LSP, for both unidirectional and bidirectional LSPs 🦋. It does not define any new procedures or protocol extensions, and it fully defers to the documents in its normative references. The clarifications can also help you interpret LSP establishment performance figures for MPLS-TE and GMPLS devices 💜.

Status of This Memo ✨

This doc is not an Internet Standards Track specification. It's published for informational purposes 📚.

This is a product of the Internet Engineering Task Force (IETF), so it reflects the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is a candidate for any level of Internet Standard; see Section 2 of RFC 5741.

Info about the current status of this doc, any errata, and how to give feedback can be found at http://www.rfc-editor.org/info/rfc6383 🫶.

Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please review those carefully, because they describe your rights and restrictions for this doc. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License ⚖️.

1. Introduction 🦋

RSVP [RFC2205] got extended to do Traffic Engineering (TE) in Multiprotocol Label Switching (MPLS) and Generalized MPLS (GMPLS) networks [RFC3209] [RFC3473]. It sets up LSPs that go through nodes and links to give you end-to-end data paths. As the signaling messages are processed, each node is programmed with "cross-connect" info, and that info tells the node how to forward the data it receives. Depending on the tech, this might mean configuring physical devices, or it might mean sending commands between different components of the node. The nature of a cross-connect is explained more in Section 1.1.1 💅.

End points of an LSP need to know when it's safe to start sending data, and "safe" has two meanings here 🔥. First, the sender needs to know that the data path is fully established, meaning the cross-connects are set up and any old, incorrect forwarding instructions are gone, so the data reaches the intended destination. Second, in optical tech, lasers must not be turned on until the correct cross-connects are in place, so service personnel aren't put at risk 🙈.

All Label Switching Routers (LSRs) along the LSP's path also need to know when to program their data planes relative to sending and receiving control-plane messages 🧚.

This doc clarifies and summarizes the RSVP-TE protocol exchanges related to programming cross-connects along an LSP, for both unidirectional and bidirectional LSPs. Bidirectional LSPs, just so you know, are only supported in GMPLS. The doc does not define any new procedures or protocol extensions, and it fully defers to the documents in its normative references 💖.

The clarifications can also help you interpret LSP establishment performance figures for MPLS-TE and GMPLS devices. For example, the dynamic provisioning performance metrics in [RFC5814] need to be understood in the context of LSP setup times, not in terms of control message exchange times. Those exchange times are only a component of the whole LSP establishment process ⭐.

Implementations could seriously benefit from this doc definitively identifying any LSR to forward the Path or Resv message [RFC3473] before programming its cross-connect. That lets them use pipelining (doing one action in the background while another one progresses) to try to minimize the total time to set up the LSP 🚀. But here's the catch 😭: while this doc gives advice and identifies the issues to consider, it's not possible to make definitive statements about how much pipelining is safe. A node can't "know" much without first probing the network (for example, with protocol extensions), and that would defeat the whole point of pipelining. Because of the number of variables introduced by path length and other node behavior, ingress might be stuck with a very pessimistic view for safety. Also, it seems unlikely that an implementation would give a full and frank description of how long it takes to program and stabilize its cross-connects 🙃. Still, this doc identifies the issues and the opportunities for pipelining in GMPLS systems 🌈.

1.1. Terminology 🎀

It's assumed you're already familiar with the basic message flows of RSVP-TE as used in MPLS-TE and GMPLS. For more details, refer to [RFC2205], [RFC3209], [RFC3471], and [RFC3473] 📚.

1.1.1. What is a Cross-Connect? 🔮

In this doc, a "cross-connection" should be taken to mean the data forwarding instructions that get installed (that is, "programmed") at a network node, or "switch" 💅.

In packet MPLS networks, this is often called the Incoming Label Map (ILM) and Next Hop Label Forwarding Entry (NHLFE) [RFC3031]. These are sometimes considered together as entries in the Label Forwarding Information Base (LFIB) [RFC4221]. Where there is admission control and resource reservation associated with the data forwarding path (such as allocating data buffers) [RFC3209], that can be treated as part of the cross-connect programming process. The reason is that the LSP won't be able to forward data the way it was agreed during the signaling protocol exchange until those resources are correctly allocated and reserved 🫶.

In non-packet networks (such as time-division multiplexing or optical switching networks), the cross-connect concept might be an electronic cross-connect array or a transparent optical device (such as a microelectromechanical system (MEMS)). In all cases, though, the concept applies to the instructions programmed into the forwarding plane (the data plane), so that incoming data for the LSP on one port can be handled correctly and forwarded out of another port 🦋.

2. Unidirectional MPLS-TE LSPs 💜

[RFC3209] describes the RSVP-TE signaling and processing for MPLS-TE packet-based networks. LSPs in these networks are unidirectional by definition, since there are no bidirectional capabilities in [RFC3209] 🌸.

Section 4.1.1.1 of [RFC3209] describes a node's process before it sends a Resv message to its upstream neighbor. It says:

      The node then sends the new LABEL object as part of the Resv
      message to the previous hop.  The node SHOULD be prepared to
      forward packets carrying the assigned label prior to sending the
      Resv message.

So the cross-connect should be in place to support traffic that might arrive at the node before the node sends the Resv. This is totally advisable 💅, because the upstream LSRs might otherwise finish their cross-connections faster and encourage the ingress to start transmitting. Then the node that sent the Resv "early" would be unable to forward the data it received, and it would be forced to drop it. Or it might accidentally send the data down the wrong LSP because of stale cross-connect info 😭.

The use of "SHOULD" [RFC2119] in that text means an implementation could be built that sends a Resv before it's ready to receive and forward data. This might happen simply because the internal construction of the node means the control-plane components can't easily tell when the cross-connection has been installed. Or it might happen because the implementation knows it will be slow and doesn't want to hold up the establishment of the LSP. In that second case, the implementation is choosing to pipeline the cross-connect programming with the protocol exchange 🚀. It's gambling that other upstream LSRs may also take some time to process, and anyway it'll be a while before the ingress actually starts sending data 🙈.

Heads up 💀: as well as the risks described above, a node that behaves like this must include a mechanism to report a failure to chase the Resv message (using a PathErr) in the event that the pipelined cross-connect processing fails.

3. GMPLS LSPs 🧚

GMPLS [RFC3945] extends RSVP-TE signaling for use in networks of different technologies [RFC3471] [RFC3473]. So RSVP-TE signaling may be used in MPLS packet switching networks, as well as layer two networks (Ethernet, Frame Relay, ATM), time-division multiplexing networks (Time Division Multiplexer (TDM), i.e., Synchronous Optical Network (SONET) and Synchronous Digital Hierarchy (SDH)), Wavelength Division Multiplexing (WDM) networks, and fiber switched network. It's giving range 🌈.

The introduction of these other technologies, especially the optical ones, brings in the second meaning of "safe" for starting data transmission that we covered in Section 1. There's a real physical safety issue here: the lasers should not be enabled until the cross-connects are correctly in place ⚡.

GMPLS supports unidirectional and bidirectional LSPs. These get split into separate sections for discussion 👑. The processing rules are inherited from [RFC3209] unless they're specifically modified by [RFC3471] and [RFC3473] 💖.

3.1. Unidirectional LSPs ✨

Okay so unidirectional LSP processing is basically the same as Section 2, with one big difference: the Suggested_Label object from [RFC3473]. 🎀 This object lets an upstream LSR "suggest" the label that its downstream neighbor should use for forward-direction data, by putting the object in a Path message. The point is to help the downstream LSR pick its label, but it also lets the upstream LSR pipeline its cross-connect programming alongside the RSVP-TE signaling. So the cross-connect might be set up before signaling is done (before the upstream LSR has gotten a Resv message carrying a Label object) 😳

So when is it actually safe to start sending data? There are three sources that answer this 👀

In particular, an ingress node should not transmit data traffic on
a suggested label until the downstream node passes a label
upstream.

So an ingress node can (safely) start transmitting once it gets a label in a Resv message. 💅

Furthermore, an ingress node SHOULD NOT transmit data traffic
using a suggested label until the downstream node passes a
corresponding label upstream.

Same idea, just confirmed again. Twice is iconic 🔥

  • Section 4.1.1.1 of [RFC3209] says:
The node then sends the new LABEL object as part of the Resv
message to the previous hop.  The node SHOULD be prepared to
forward packets carrying the assigned label prior to sending the
Resv message.

The word "prior" is doing a LOT of work here 🧚 It means the cross-connect has to be in place for forward traffic before the Resv goes out. Every transit node and the egress node need to finish their cross-connects before they send the Resv to their upstream neighbors.

So, same as in Section 2, the ingress must not start transmitting until it has both received a Resv and programmed its own cross-connect. 🫶

3.2. Bidirectional LSPs 🦋

A bidirectional LSP gets set up with one signaling exchange: a Path message goes from ingress to egress, and a Resv comes back from egress to ingress. The LSP itself is made of two sets of forwarding state. One handles the path from ingress to egress (the forwards data path), and the other handles the path from egress to ingress (the reverse data path). Two lanes, one highway 🛣️✨

3.2.1. Forwards Direction Data

Forwards direction data works exactly like a unidirectional LSP, as described in Section 3.1. Easy peasy 🌸

3.2.2. Reverse Direction Data

For the reverse direction, each LSR sends an Upstream_Label object in the Path message to its downstream neighbor. That object tells the downstream LSR which label to use for data going back to the upstream LSR (that's the reverse direction data). The downstream LSR confirms the label when it sends a Resv message. There's no explicit confirmation of the label in the Resv, but if the downstream LSR didn't like the label, it would send a PathErr message instead 🙈

Now the upstream LSR has to figure out timing. Specifically, it has to decide when to send the Path message compared to when it programs its cross-connect. Its options are:

  • Program the cross-connect before it sends the Path message.
  • Overlap the programming with the message exchange.
  • Wait until it receives a Resv from its downstream neighbor.

The defining reference is Section 3.1 of [RFC3473]:

The Upstream_Label object MUST indicate a label that is valid for
forwarding at the time the Path message is sent.

"Valid for forwarding" here means it's safe for the LSR that sent the Path message to receive data, and that this LSR will forward data correctly. It does not mean the label is "acceptable for use," meaning the label is free to be cross-connected. Big difference, bestie 💅

This gets clarified later in Section 3.1 of [RFC3473]:

Terminator nodes process Path messages as usual, with the
exception that the upstream label can immediately be used to
transport data traffic associated with the LSP upstream towards
the initiator.

This is a clear statement: once an egress node has fully processed a Path message, it's completely safe to send data back toward the ingress (reverse direction data) 🌈

From all that, we can deduce a few things:

  • An LSR must not wait to receive a Resv message before it programs the cross-connect for reverse direction data. It has to be ready to receive data from the moment the egress finishes processing the Path message it gets (so before the egress sends a Resv back upstream).
  • An LSR may expect to start receiving reverse direction data as soon as it sends a Path message for a bidirectional LSP.
  • An LSR may make some assumptions about the time gap between sending a Path message and that message reaching and being processed by the egress. It can use that gap to pipeline its cross-connect programming. ⏳✨

3.3. ResvConf Message 💌

The ResvConf message is used in standard RSVP [RFC2205] to let the ingress confirm to the egress that the Resv was received successfully, and what bandwidth got reserved. In RSVP-TE [RFC3209] and GMPLS [RFC3473], bandwidth isn't expected to change along the LSP's path, so the ResvConf's job is basically just confirming that the LSP was set up successfully 🎀

The egress can ask for a ResvConf by including a Resv_Confirm object in the Resv message it sends. When the ingress gets that Resv and sees the Resv_Confirm object, it can reply with a ResvConf message 💖

This mechanism could be a doubly secure way for the egress to make sure the reverse direction data path is safely in place before it starts transmitting. If the egress waits for a ResvConf message, it can be sure the whole LSP is in place. 🔒

BUT (and this is a big but) this mechanism is overkill given what Section 3.2.2 already defines, and it would delay LSP setup by one full end-to-end message propagation cycle 🙃 Also, the ResvConf message isn't guaranteed to be generated. And honestly, many (if not most) GMPLS implementations neither request nor send ResvConf messages. So an egress relying on receiving a ResvConf to know it's safe to start transmitting reverse direction data is not recommended. Not it, sis 💀

3.4. Administrative Status 🧚‍♀️

GMPLS has one more tool for keeping the LSP safe. The Administrative Status information is defined in Section 8 of [RFC3471], and it gets carried in the Admin_Status Object defined in Section 7 of [RFC3473] 🍓

This object lets an ingress set up an LSP in the "Administratively Down" state. According to [RFC3471], this state means:

... the local actions related to the "administratively down" state
should be taken.

In this state, the LSP is assumed to exist (so the cross-connects are all in place), but no data gets transmitted. For example, in optical systems, the lasers are off 🔦🚫

The Admin_Status object can also put the LSP into the "Testing" state. According to [RFC3471], that means:

... the local actions related to the "testing" mode should be
taken.

This state lets you test the LSP's connectivity without actually sending user data. For instance, in an optical system, you could run a data continuity test (with some external coordination for errors). In a packet network, you could use a connection verification exchange, like the in-band Virtual Circuit Connectivity Verification described in Section 5.1.1 of [RFC5085]. Once connectivity is verified, the LSP can be switched to active mode, and user data can start flowing 🚀

These processes matter most in systems where the control-plane processors are physically separate from the data-plane cross-connects (like when a communication protocol runs between the control-plane processor and the data-plane switch). In that setup, successful control-plane signaling can't necessarily be taken as proof that the data plane got programmed correctly 👀

4. Implications for Performance Metrics 📊

How fast an LSR can handle and forward control-plane messages, and how fast it can program its cross-connects, varies a LOT from device to device. It depends on the switching technology, the control-plane connectivity, and the implementation. All of this affects how quickly an LSP can be set up 🐢⚡

Different applications need different LSP setup speeds, and this can matter a ton in recovery scenarios. Service providers thinking about deploying MPLS-TE or GMPLS equipment need a good benchmark for how that equipment performs. Equipment vendors also need to be compared on a level playing field 🏁

To provide a basis for comparison, [RFC5814] defines a set of performance metrics for evaluating dynamic LSP provisioning performance in MPLS-TE/GMPLS networks. But anyone using these metrics has to be careful about what they're actually measuring. It isn't enough to know that a control-plane message got processed and forwarded: the cross-connect has to be in place before the LSP can be used. So you have to make sure devices really follow the procedures clarified in Section 2 of this document, and aren't just forwarding control-plane messages while quietly programming the cross-connects in the background. No cheating, bestie 😤

5. Security Considerations 🔐

This document does not define any network behavior, and it doesn't introduce or try to solve any security issues.

That said, a clear understanding of when to start sending data may reduce the risk of data being accidentally delivered to the wrong place, or of people getting hurt. ⚠️ Please take this seriously, it's not a drill 🚨

6. Acknowledgments 🫶

Thanks to Weiqiang Sun, Olufemi Komolafe, Daniel King, and Stewart Bryant for their review and comments. You're all literally the best 💜✨

7. References 📚

7.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
          Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S.
          Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1
          Functional Specification", RFC 2205, September 1997.

[RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V.,
          and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP
          Tunnels", RFC 3209, December 2001.

[RFC3471] Berger, L., Ed., "Generalized Multi-Protocol Label
          Switching (GMPLS) Signaling Functional Description", RFC
          3471, January 2003.

[RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label
          Switching (GMPLS) Signaling Resource ReserVation Protocol-
          Traffic Engineering (RSVP-TE) Extensions", RFC 3473,
          January 2003.

[RFC3945] Mannie, E., Ed., "Generalized Multi-Protocol Label
          Switching (GMPLS) Architecture", RFC 3945, October 2004.

7.2. Informative References ✨

okay bestie, these are the "for background, go read this" references 📚 Nothing here is required to implement the spec, but they're the good stuff for understanding the context 🧠

[RFC3031] Rosen, E., Viswanathan, A., and R. Callon, "Multiprotocol
          Label Switching Architecture", RFC 3031, January 2001.

[RFC4221] Nadeau, T., Srinivasan, C., and A. Farrel, "Multiprotocol
          Label Switching (MPLS) Management Overview", RFC 4221,
          November 2005.

[RFC5085] Nadeau, T., Ed., and C. Pignataro, Ed., "Pseudowire Virtual
          Circuit Connectivity Verification (VCCV): A Control Channel
          for Pseudowires", RFC 5085, December 2007.

[RFC5814] Sun, W., Ed., and G. Zhang, Ed., "Label Switched Path (LSP)
          Dynamic Provisioning Performance Metrics in Generalized
          MPLS Networks", RFC 5814, March 2010.

Authors' Addresses 💌

Meet the two authors who made this happen 🫶

Kohei Shiomoto
NTT Service Integration Laboratories
3-9-11 Midori
Musashino, Tokyo 180-8585
Japan
Phone: +81 422 59 4402
EMail: shiomoto.kohei@lab.ntt.co.jp

Adrian Farrel
Old Dog Consulting
EMail: adrian@olddog.co.uk

That's the whole RFC, bestie! 🌈✨ Kohei and Adrian literally wrote the rules for how GMPLS LSPs behave, and now you know all about it 👑💅

📦 where this page comes from