RFC 8456
Benchmarking Methodology for Software-Defined Networking (SDN) Controller Performance
RFC 8456 🎀 Title: Benchmarking Methodology for Software-Defined Networking (SDN) Controller Performance 💅
Okay bestie, this one is an Informational doc, so it's published for information and is not a standards-track spec 🌸 It came out in October 2018 and has the series number Request for Comments: 8456. The ISSN is 2070-1721. The authors are V. Bhuvaneswaran and A. Basil from Veryx Technologies, M. Tassinari from Hewlett Packard Enterprise, V. Manral from NanoSec, and S. Banks from VSS Monitoring 🫶 It doesn't list anything it obsoletes or updates, so it's standing on its own ✨
Abstract 🦋
This doc gives you methodologies for benchmarking the control-plane performance of SDN Controllers 💜 The SDN Controller is the core piece of the SDN architecture that decides how the network behaves 🧚 Controllers come in lots of different designs because they all have to hit different network functionality goals. So the authors treat the SDN Controller as a black box and write the method so it doesn't care which protocols or network services a controller supports. The result is a way to measure the performance of all controller implementations 🔥
Status of This Memo 🌈
This document is not an Internet Standards Track specification; it is published for informational purposes.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 7841.
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8456.
Copyright Notice ⭐
Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Table of Contents 🍓
1. Introduction ....................................................4
1.1. Conventions Used in This Document ..........................4
2. Scope ...........................................................4
3. Test Setup ......................................................4
3.1. Test Setup - Controller Operating in Standalone Mode .......5
3.2. Test Setup - Controller Operating in Cluster Mode ..........6
4. Test Considerations .............................................7
4.1. Network Topology ...........................................7
4.2. Test Traffic ...............................................7
4.3. Test Emulator Requirements .................................7
4.4. Connection Setup ...........................................8
4.5. Measurement Point Specification and Recommendation .........9
4.6. Connectivity Recommendation ................................9
4.7. Test Repeatability .........................................9
4.8. Test Reporting .............................................9
5. Benchmarking Tests .............................................11
5.1. Performance ...............................................11
5.1.1. Network Topology Discovery Time ....................11
5.1.2. Asynchronous Message Processing Time ...............13
5.1.3. Asynchronous Message Processing Rate ...............14
5.1.4. Reactive Path Provisioning Time ....................17
5.1.5. Proactive Path Provisioning Time ...................19
5.1.6. Reactive Path Provisioning Rate ....................21
5.1.7. Proactive Path Provisioning Rate ...................23
5.1.8. Network Topology Change Detection Time .............25
5.2. Scalability ...............................................26
5.2.1. Control Sessions Capacity ..........................26
5.2.2. Network Discovery Size .............................27
5.2.3. Forwarding Table Capacity ..........................29
5.3. Security ..................................................31
5.3.1. Exception Handling .................................31
5.3.2. Handling Denial-of-Service Attacks .................32
5.4. Reliability ...............................................34
5.4.1. Controller Failover Time ...........................34
5.4.2. Network Re-provisioning Time .......................36
6. IANA Considerations ............................................37
7. Security Considerations ........................................38
8. References .....................................................38
8.1. Normative References ......................................38
8.2. Informative References ....................................38
Appendix A. Benchmarking Methodology Using OpenFlow Controllers ...39
A.1. Protocol Overview ..........................................39
A.2. Messages Overview ..........................................39
A.3. Connection Overview ........................................39
A.4. Performance Benchmarking Tests .............................40
A.4.1. Network Topology Discovery Time ........................40
A.4.2. Asynchronous Message Processing Time ...................42
A.4.3. Asynchronous Message Processing Rate ...................43
A.4.4. Reactive Path Provisioning Time ........................44
A.4.5. Proactive Path Provisioning Time .......................46
A.4.6. Reactive Path Provisioning Rate ........................47
A.4.7. Proactive Path Provisioning Rate .......................49
A.4.8. Network Topology Change Detection Time .................50
A.5. Scalability ................................................51
A.5.1. Control Sessions Capacity ..............................51
A.5.2. Network Discovery Size .................................52
A.5.3. Forwarding Table Capacity ..............................54
A.6. Security ...................................................55
A.6.1. Exception Handling .....................................55
A.6.2. Handling Denial-of-Service Attacks .....................57
A.7. Reliability ................................................59
A.7.1. Controller Failover Time ...............................59
A.7.2. Network Re-provisioning Time ...........................61
Acknowledgments ...................................................63
Authors' Addresses ................................................641. Introduction ✨
This doc gives generic methodologies for benchmarking SDN Controller performance 💖 To get the functionality they want, controllers may support lots of northbound and southbound protocols, run a wide range of applications, and work alone or as part of a group 🧚 This doc treats the SDN Controller as a black box, no matter how it's designed or built 🔥 The tests here can benchmark a controller for performance, scalability, reliability, and security, all independently of northbound and southbound protocols. The terminology for benchmarking SDN Controllers lives in the companion terminology document [RFC8455] 📚 These tests can run on an SDN Controller that's a virtual machine (VM) instance or on a bare metal server. This doc is for anyone who wants to measure an SDN Controller's performance or compare the performance of different SDN Controllers 💅
1.1. Conventions Used in This Document 🌸
Okay, the key words here are "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL". They get read as described in BCP 14 [RFC2119] [RFC8174], but only when they show up in all capitals, just like this 🫶
2. Scope 🦋
This doc defines a way to measure the networking metrics of SDN Controllers 💜 For this memo, an SDN Controller is a function that manages and controls Network Devices. Any SDN Controller without a control capability is out of scope here 🙈 The tests in this doc let you benchmark SDN Controllers in two ways: standalone mode (a standalone controller) and cluster mode (a cluster of homogeneous controllers) ✨ These tests are recommended for lab environments rather than live network deployments 🍓 Performance benchmarking of a federation of controllers (i.e., a set of SDN Controllers) managing different domains is beyond the scope of this document 🌈
3. Test Setup 🎀
As noted above, the tests in this doc let you measure an SDN Controller's performance in standalone mode and cluster mode 💖 This section defines common reference topologies that the individual tests described later in this document refer back to 🧚
3.1. Test Setup - Controller Operating in Standalone Mode 🎀
okay so bestie, first up is the standalone setup 💅 picture a big ol' box up top that's the Application-Plane Test Emulator 🌸, holding an Application and a Service. it talks down through the (I2) interface, which is the Northbound Interface 🦋
right under that is the Device Under Test (DUT) 👑, and inside it lives one lonely SDN Controller (standalone, so it's solo ✨). the DUT then talks south through the Southbound Interface to the (I1) part, where the Forwarding-Plane Test Emulator is hanging out 🧚
down there you've got a whole web of Network Devices (Device 1, Device 2, all the way to Device n - 1 and Device n) linked up with the l0 and ln links, plus two Test Traffic Generators, (TP1) and (TP2) 💖 TP1 hooks into Device 1 and TP2 hooks into Device n
+-----------------------------------------------------------+
| Application-Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
| (Northbound Interface)
+-------------------------------+
| +----------------+ |
| | SDN Controller | |
| +----------------+ |
| |
| Device Under Test (DUT) |
+-------------------------------+
| (Southbound Interface)
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-------------+ |
| | Network | | Network | |
| | Device 2 |--..-| Device n - 1| |
| +-----------+ +-------------+ |
| / \ / \ |
| / \ / \ |
| l0 / X \ ln |
| / / \ \ |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 1 |..| Device n | |
| +-----------+ +-----------+ |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding-Plane Test Emulator |
+-----------------------------------------------------------+
Figure 13.2. Test Setup - Controller Operating in Cluster Mode 🌈
same vibe, but this time the controller isn't solo, it's a whole squad 🫶 inside the DUT you've got SDN Controller 1 and SDN Controller n chatting with each other over the E/W link 💜 everything else is the same as before: the Application-Plane Test Emulator up top on (I2), the Northbound Interface, the Southbound Interface, the (I1) network full of Network Devices, and the Forwarding-Plane Test Emulator with TP1 and TP2 💅
+-----------------------------------------------------------+
| Application-Plane Test Emulator |
| |
| +-----------------+ +-------------+ |
| | Application | | Service | |
| +-----------------+ +-------------+ |
| |
+-----------------------------+(I2)-------------------------+
|
| (Northbound Interface)
+---------------------------------------------------------+
| |
| +------------------+ +------------------+ |
| | SDN Controller 1 | <--E/W--> | SDN Controller n | |
| +------------------+ +------------------+ |
| |
| Device Under Test (DUT) |
+---------------------------------------------------------+
| (Southbound Interface)
|
+-----------------------------+(I1)-------------------------+
| |
| +-----------+ +-------------+ |
| | Network | | Network | |
| | Device 2 |--..-| Device n - 1| |
| +-----------+ +-------------+ |
| / \ / \ |
| / \ / \ |
| l0 / X \ ln |
| / / \ \ |
| +-----------+ +-----------+ |
| | Network | | Network | |
| | Device 1 |..| Device n | |
| +-----------+ +-----------+ |
| | | |
| +---------------+ +---------------+ |
| | Test Traffic | | Test Traffic | |
| | Generator | | Generator | |
| | (TP1) | | (TP2) | |
| +---------------+ +---------------+ |
| |
| Forwarding-Plane Test Emulator |
+-----------------------------------------------------------+
Figure 24. Test Considerations 🧠
4.1. Network Topology 🕸️
okay so the test cases SHOULD use a Leaf-Spine topology with at least two Network Devices in the topology for benchmarking 🌸 Test traffic generators TP1 and TP2 SHOULD be connected to the leaf Network Device 1 and the leaf Network Device n 💅 it's also recommended that you benchmark the controller for many network topologies and a varying number of Network Devices, so you get a complete performance picture ✨ and ngl, be careful: when the topology has redundant network paths, make sure a loop-prevention mechanism is enabled in either the SDN Controller or the network 🙈
4.2. Test Traffic 🚦
test traffic is how you tell the controller about the asynchronous arrival of new flows 🦋 the test cases SHOULD use frame sizes of 128, 512, and 1508 bytes for benchmarking 💖 tests using jumbo frames are OPTIONAL 🎀
Wait, correction for accuracy: the requirement keywords in this section are lowercase in the source, so they stay as written. The test cases SHOULD use frame sizes of 128, 512, and 1508 bytes, and tests using jumbo frames are optional. Keeping it exact, bestie 🫶
4.3. Test Emulator Requirements 📏
the test emulator SHOULD timestamp the transmitted and received control messages to and from the controller on the established network connections 🕰️ the test cases use those values to compute the controller processing time ⭐
4.4. Connection Setup 🔌
okay this one's giving options 🌈 there may be controller implementations that support unencrypted and encrypted network connections with Network Devices 💜 further, the controller may be backward compatible with Network Devices running older versions of southbound protocols 🦋 it may be useful to measure the controller's performance with one or more of the connection setup methods below 💅 for cases with encrypted communications between the controller and the switch, key management and key exchange MUST take place before any performance or benchmark measurements 🔐
- Unencrypted connection with Network Devices, running the same protocol version.
- Unencrypted connection with Network Devices, running different protocol versions.
Examples:
a. Controller running current protocol version and switch running older protocol version.
b. Controller running older protocol version and switch running current protocol version.
- Encrypted connection with Network Devices, running the same protocol version.
- Encrypted connection with Network Devices, running different protocol versions.
Examples:
a. Controller running current protocol version and switch running older protocol version.
b. Controller running older protocol version and switch running current protocol version.
4.5. Measurement Point Specification and Recommendation 📍
the accuracy of your measurements depends on a bunch of factors, including where the observations get captured 👀 for example, the notification can be observed at the controller or at the test emulator 💖 the test operator SHOULD make the observations and measurements at the interfaces of the test emulator, unless a specific test explicitly says otherwise ✨ in any case, the locations of measurement points MUST be reported 📝
4.6. Connectivity Recommendation 🔗
the SDN Controller in the test setup SHOULD be connected directly with the forwarding-plane and management-plane test emulators 🫶 that way you avoid any delays or failures from intermediate devices during benchmarking tests 🙅♀️ when the controller is implemented as a virtual machine, details of the physical and logical connectivity MUST be reported 💻
4.7. Test Repeatability 🔁
to build confidence in your results, it is recommended that each test SHOULD be repeated a minimum of 10 times 🌟
4.8. Test Reporting 📋
every test has a reporting format with some global, identical components and some components specific to that individual test 🎀 the following parameters for test configuration and controller settings MUST be reflected in the test report 💅
Test Configuration Parameters:
- Controller name and version
- Northbound protocols and versions
- Southbound protocols and versions
- Controller redundancy mode (standalone or cluster mode)
- Connection setup (unencrypted or encrypted)
- Network Device type (physical, virtual, or emulated)
- Number of nodes
- Number of links
- Data-plane test traffic type
- Controller system configuration (e.g., physical or virtual machine, CPU, memory, caches, operating system, interface speed, storage)
- Reference test setup (e.g., the setup shown in Section 3.1)
Parameters for Controller Settings:
- Topology rediscovery timeout
- Controller redundancy mode (e.g., active-standby)
- Controller state persistence enabled/disabled
to make sure the test is repeatable, the following capabilities of the test emulator SHOULD be reported 🔍
- Maximum number of Network Devices that the forwarding plane emulates
- Control message processing time (e.g., topology discovery messages)
one way to determine the above two values is to simulate the required control sessions and messages from the control plane 🧚
5. Benchmarking Tests 🏁
5.1. Performance ⚡
5.1.1. Network Topology Discovery Time ✨
Objective:
okay so this one measures how long the controller(s) take to figure out the whole network topology 🗺️ 🫶 It starts at the very first discovery message the controller sends out through its southbound interface and ends when every feature of the static topology has been found. main character energy, literally.
Reference Test Setup:
this test SHOULD use one of the test setups from Section 3.1 or Section 3.2 of this document 💅
Prerequisites:
- The controller MUST support network discovery. no cap, this is the entry ticket 🎟️
- The tester should be able to pull up the discovered topology through either the controller's management interface or its northbound interface, so they can check whether discovery worked and finished. 👀
- Make sure the controller's topology rediscovery timeout is set to the maximum value so it doesn't start rediscovering in the middle of the test 🙈
Procedure:
- Make sure the controller is operational and its network applications, northbound interface, and southbound interface are all up and running ✨
- Set up the network connections between the controller and the Network Devices. 🔌
- Record the time of the first discovery message (Tm1) that the controller sends, as seen at the forwarding-plane test emulator interface (I1). ⏰
- Query the controller every t seconds (the RECOMMENDED value for t is 3) to get the discovered topology through the northbound interface or the management interface, then compare it with the deployed topology. the RECOMMENDED value is 3, so that's the vibe 🎀
- Stop the trial when the discovered topology matches the deployed network topology, or when the discovered topology returns the same details for three consecutive queries. 🛑
- Record the time of the last discovery message (Tmn) sent to the controller, as seen at the forwarding-plane test emulator interface (I1), when the trial finishes successfully (for example, when the topology matches). 🏁
Measurements:
Topology Discovery Time (DT1) = Tmn - Tm1
DT1 + DT2 + DT3 .. DTn
Average Topology Discovery Time (TDm) = -----------------------
Total Trials
SUM[SQUAREOF(DTi - TDm)]
Topology Discovery Time Variance (TDv) = ------------------------
Total Trials - 1Reporting Format:
The Topology Discovery Time results MUST be reported in tabular format, with a row for each successful iteration. The last row of the table shows the Topology Discovery Time variance, and the row right above it shows the Average Topology Discovery Time. 📊💖
If this test is repeated with a varying number of nodes over the same topology, the results SHOULD be reported in the form of a graph. The X coordinate SHOULD be the number of nodes (N), and the Y coordinate SHOULD be the Average Topology Discovery Time. graphs are so iconic when they tell a story 📈✨
5.1.2. Asynchronous Message Processing Time 🦋
Objective:
this one measures how long the controller(s) take to process an asynchronous message. It starts when a Network Device sends an asynchronous message after all the devices have been discovered by the controller(s), and it ends when the controller(s) send back a response message at their southbound interface. 🌸
Reference Test Setup:
this test SHOULD use one of the test setups from Section 3.1 or Section 3.2 of this document. same setups, we're consistent bestie 🫶
Prerequisite:
The controller MUST have successfully completed the network topology discovery for the connected Network Devices. no discovery, no party 🚫🎉
Procedure:
- Generate asynchronous messages from every connected Network Device to the SDN Controller, one at a time in series, from the forwarding-plane test emulator for the Trial Duration. one at a time, in order, lowkey so satisfying 🔁
- Record every request transmit time (T1) and the matching response received time (R1) at the forwarding-plane test emulator interface (I1) for every successful message exchange. ⏱️
Measurements:
Asynchronous Message Processing Time (APT1) =
SUM{Ri} - SUM{Ti}
-----------------------
Nrx
Where Nrx is the total number of successful messages exchanged.
Average Asynchronous Message Processing Time =
APT1 + APT2 + APT3 .. APTn
--------------------------
Total Trials
Asynchronous Message Processing Time Variance (TAMv) =
SUM[SQUAREOF(APTi - TAMm)]
--------------------------
Total Trials - 1
Where TAMm is the Average Asynchronous Message Processing Time.Reporting Format:
The Asynchronous Message Processing Time results MUST be reported in tabular format, with a row for each iteration. The last row of the table shows the Asynchronous Message Processing Time variance, and the row above it shows the Average Asynchronous Message Processing Time. 💅📋
The report SHOULD capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Successful messages exchanged (Nrx) ✅
- Percentage of unsuccessful messages exchanged, computed using the formula
((1 - Nrx/Ntx) * 100), where Ntx is the total number of messages transmitted to the controller 📉
If this test is repeated with a varying number of nodes with the same topology, the results SHOULD be reported in the form of a graph. The X coordinate SHOULD be the number of nodes (N), and the Y coordinate SHOULD be the Average Asynchronous Message Processing Time. 🌈
5.1.3. Asynchronous Message Processing Rate 🍓
Objective:
this one measures how many responses to asynchronous messages (like a new flow arrival notification message, a link down, and so on) the controller(s) actually processed, and replied to with a valid and productive (non-trivial) response message. 🧚
Using a single procedure, this test measures the following two benchmarks on the Asynchronous Message Processing Rate (see Section 2.3.1.3 of [RFC8455]):
- Maximum Asynchronous Message Processing Rate
- Loss-Free Asynchronous Message Processing Rate
so it's two benchmarks in one trip, efficient and iconic 🔥 Here, both benchmarks come from a series of trials where the messages sent to the controller(s) and the responses received from them are counted over the Trial Duration. The message response rate and the Message Loss Ratio are calculated for each trial. 📊✨
Reference Test Setup:
this test SHOULD use one of the test setups from Section 3.1 or Section 3.2 of this document. 💖
Prerequisites:
- The controller(s) MUST have successfully completed the network topology discovery for the connected Network Devices. discovery first, then the fun begins 🎀
- Choose and record the Trial Duration (Td), the sending rate STEP size, the tolerance on equality for two consecutive trials (P%), and the maximum possible message-sending rate (
Ntx1/Td). write all of this down, no skipping 📝
Procedure:
- Generate asynchronous messages continuously, at the maximum possible rate, on the established connections from all the emulated/simulated Network Devices for the given Trial Duration (Td). full speed, no brakes 🏎️💨
- Record the total number of responses received (Nrx1) from the controller, as well as the number of messages sent (Ntx1) to the controller within the Trial Duration (Td). 🧾
- Calculate the Asynchronous Message Processing Rate (APR1) and the Message Loss Ratio (Lr1). Make sure the controller(s) have gone back to normal operation first. breathe, controller, breathe 🧘
- Repeat the trial, this time reducing the asynchronous message-sending rate from the last trial by the STEP size. dial it down a little 🔽
- Keep repeating the trials and lowering the sending rate until both the maximum value of Nrxn (the number of responses received from the controller) and the Nrxn that matches a Loss Ratio of zero have been found. we're hunting for the sweet spots 🎯
- The trials for the benchmark levels MUST be repeated using the same asynchronous message rates until the responses received from the controller are equal (+/-P%) for two consecutive trials. consistency is everything 💜
- Record the number of responses received (Nrxn) from the controller, as well as the number of messages sent (Ntxn) to the controller, in the last trial. 📌
Measurements:
Nrxn
Asynchronous Message Processing Rate (APRn) = -----
Td
Maximum Asynchronous Message Processing Rate = MAX(APRn) for all n
Nrxn
Asynchronous Message Loss Ratio (Lrn) = 1 - -----
Ntxn
Loss-Free Asynchronous Message Processing Rate = MAX(APRn)
given Lrn = 0Reporting Format:
The Asynchronous Message Processing Rate results MUST be reported in tabular format, with a row for each trial. 📊
The table should report the following information, in addition to the configuration parameters captured per Section 4.8, with these columns:
- Offered rate (Ntxn/Td)
- Asynchronous Message Processing Rate (APRn)
- Loss Ratio (Lr)
- Benchmark at this iteration (blank for none, Maximum Asynchronous Message Processing Rate, Loss-Free Asynchronous Message Processing Rate)
The results MAY be presented in the form of a graph. The X axis SHOULD be the offered rate, and dual Y axes would represent the Asynchronous Message Processing Rate and the Loss Ratio, respectively. two axes, double the drama 📈📉🫶
If this test is repeated with a varying number of nodes over the same topology, the results SHOULD be reported in the form of a graph. The X axis SHOULD be the number of nodes (N), and the Y axis SHOULD be the Asynchronous Message Processing Rate. Both the Maximum Asynchronous Message Processing Rate and the Loss-Free Asynchronous Message Processing Rate should be plotted for each N. the graphs are giving everything 🌈💅
5.1.4. Reactive Path Provisioning Time 🛤️✨
okay so bestie, this one is about how long the controller takes to set up a whole path reactively, meaning the network asks for help when a packet shows up with no rule for it 🙈. The clock starts at the very first flow provisioning request the controller gets on its southbound interface and stops at the very last flow provisioning response it sends back out that same interface. Main character energy for the controller, no cap 💅
Objective:
Measure the time taken by the controller to set up a path reactively between source and destination nodes, defined as the interval starting with the first flow provisioning request message received by the controller(s) at its southbound interface and ending with the last flow provisioning response message sent from the controller(s) at its southbound interface.
Reference Test Setup:
okay this test SHOULD use one of the test setups from Section 3.1 or Section 3.2 of this document 🎀. The number of Network Devices in the path is a parameter you can vary, going from two all the way up to the maximum discovery size, across repeated runs of the test 🦋.
Prerequisites:
- The controller MUST contain the network topology information for the deployed network topology. 🌸
- The controller should know the location of the destination endpoint for which the path has to be provisioned. This can be achieved through dynamic learning or static provisioning. 💜
- Ensure that the default action for "flow miss" in the Network Device is configured to "send to controller". 🧚
- Ensure that each Network Device in a path requires the controller to make the forwarding decision while paving the entire path. 🫶
Procedure:
- Send a single traffic stream from test traffic generator TP1 to test traffic generator TP2. ✨
- Record the time of the first flow provisioning request message sent to the controller (Tsf1) from the Network Device at the forwarding-plane test emulator interface (I1). 📝
- Wait for the arrival of the first traffic frame at the endpoint (i.e., test traffic generator TP2) or the expiry of the Trial Duration (Td). ⏳
- Record the time of the last flow provisioning response message received from the controller (Tdf1) to the Network Device at the forwarding-plane test emulator interface (I1). 🎯
Measurements:
Reactive Path Provisioning Time (RPT1) = Tdf1 - Tsf1
Average Reactive Path Provisioning Time =
RPT1 + RPT2 + RPT3 .. RPTn
--------------------------
Total Trials
Reactive Path Provisioning Time Variance (TRPv) =
SUM[SQUAREOF(RPTi - TRPm)]
--------------------------
Total Trials - 1
Where TRPm is the Average Reactive Path Provisioning Time.Reporting Format:
The Reactive Path Provisioning Time results MUST be reported in tabular format, with a row for each iteration. 📊 The last row of the table indicates the Reactive Path Provisioning Time variance, and the previous row indicates the Average Reactive Path Provisioning Time. 💖
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path 🌈
5.1.5. Proactive Path Provisioning Time 🦄
okay this one flips the script 🔥. Instead of waiting for packets to trigger things, the path is set up proactively, so the clock starts when the first proactive flow gets provisioned on the controller's northbound interface and stops at the last flow provisioning response sent out southbound. It's giving planning ahead, iconic 👑.
Objective:
Measure the time taken by the controller to set up a path proactively between source and destination nodes, defined as the interval starting with the first proactive flow provisioned in the controller(s) at its northbound interface and ending with the last flow provisioning response message sent from the controller(s) at its southbound interface.
Reference Test Setup:
This test SHOULD use one of the test setups from Section 3.1 or Section 3.2 of this document. 🎀
Prerequisites:
- The controller MUST contain the network topology information for the deployed network topology. 🌸
- The controller should know the location of the destination endpoint for which the path has to be provisioned. This can be achieved through dynamic learning or static provisioning. 💜
- Ensure that the default action for "flow miss" in the Network Device is "drop". 🙅♀️
Procedure:
- Send a single traffic stream from test traffic generator TP1 to test traffic generator TP2. ✨
- Install the flow entries so that the traffic travels from test traffic generator TP1 until it reaches test traffic generator TP2 through the controller's northbound interface or management interface. 🧚
- Wait for the arrival of the first traffic frame at test traffic generator TP2 or the expiry of the Trial Duration (Td). ⏳
- Record the time when the proactive flow is provisioned in the controller (Tsf1) at the management-plane test emulator interface (I2). 📝
- Record the time of the last flow provisioning message received from the controller (Tdf1) at the forwarding-plane test emulator interface (I1). 🎯
Measurements:
Proactive Flow Provisioning Time (PPT1) = Tdf1 - Tsf1
Average Proactive Path Provisioning Time =
PPT1 + PPT2 + PPT3 .. PPTn
--------------------------
Total Trials
Proactive Path Provisioning Time Variance (TPPv) =
SUM[SQUAREOF(PPTi - TPPm)]
--------------------------
Total Trials - 1
Where TPPm is the Average Proactive Path Provisioning Time.Reporting Format:
The Proactive Path Provisioning Time results MUST be reported in tabular format, with a row for each iteration. 📊 The last row of the table indicates the Proactive Path Provisioning Time variance, and the previous row indicates the Average Proactive Path Provisioning Time. 💖
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path 🌈
5.1.6. Reactive Path Provisioning Rate 🌟
last one in this batch and it's a rate test, so we're counting how many independent paths the controller can set up per second reactively 💅. It's measured on the southbound side, counting the flow provisioning requests that turn into paths over the Trial Duration. Think of it like a bakery counting how many cupcakes come out of the oven every minute 🧁.
Objective:
Measure the maximum number of independent paths a controller can concurrently establish per second between source and destination nodes reactively, defined as the number of paths provisioned per second by the controller(s) at its southbound interface for the flow provisioning requests received for path provisioning at its southbound interface between the start of the test and the expiry of the given Trial Duration.
Reference Test Setup:
This test SHOULD use one of the test setups from Section 3.1 or Section 3.2 of this document. 🎀
Prerequisites:
- The controller MUST contain the network topology information for the deployed network topology. 🌸
- The controller should know the location of destination addresses for which the paths have to be provisioned. This can be achieved through dynamic learning or static provisioning. 💜
- Ensure that the default action for "flow miss" in the Network Device is configured to "send to controller". 🧚
- Ensure that each Network Device in a path requires the controller to make the forwarding decision while provisioning the entire path. 🫶
Procedure:
- Send traffic with unique source and destination addresses from test traffic generator TP1. ✨
- Record the total number of unique traffic frames (Ndf) received at test traffic generator TP2 within the Trial Duration (Td). 📝
Measurements:
Ndf
Reactive Path Provisioning Rate (RPR1) = ------
Td
Average Reactive Path Provisioning Rate =
RPR1 + RPR2 + RPR3 .. RPRn
--------------------------
Total Trials
Reactive Path Provisioning Rate Variance (RPPv) =
SUM[SQUAREOF(RPRi - RPPm)]
--------------------------
Total Trials - 1
Where RPPm is the Average Reactive Path Provisioning Rate.Reporting Format:
The Reactive Path Provisioning Rate results MUST be reported in tabular format, with a row for each iteration. 📊 The last row of the table indicates the Reactive Path Provisioning Rate variance, and the previous row indicates the Average Reactive Path Provisioning Rate. 💖
The report should capture the following information, in addition to the configuration parameters captured per Section 4.8:
- Number of Network Devices in the path 🌈
- Offered rate 🦋
5.1.7. Proactive Path Provisioning Rate 🚀
Objective:
- Okay bestie, this one measures the most independent paths a controller can set up per second proactively 🛤️ That means paths it provisions ahead of time, based on what it was asked for through its northbound interface. The count is how many paths get provisioned per second by the controller(s) at its southbound interface, measured from the start of the test until the Trial Duration runs out. The measurement is based on data-plane observations of successful path activation 👀
Reference Test Setup:
- This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🎀
Prerequisites:
- The controller MUST contain the network topology information for the deployed network topology. 🗺️
- The controller should know the location of destination addresses for which the paths have to be provisioned. It can learn these dynamically or get them through static provisioning 🧭
- Make sure the default action for "flow miss" in the Network Device is "drop" 🙅♀️
Procedure:
- Send traffic continuously with unique source and destination addresses from test traffic generator TP1 🌊
- Install the matching flow entries so the traffic travels from the simulated sources at TP1 all the way to the simulated destinations at test traffic generator TP2, going through the controller's northbound interface or management interface 🦋
- Record the total number of unique traffic frames (Ndf) received at TP2 within the Trial Duration (Td) 📝
Measurements:
Ndf
Proactive Path Provisioning Rate (PPR1) = ------
Td
Average Proactive Path Provisioning Rate =
PPR1 + PPR2 + PPR3 .. PPRn
--------------------------
Total Trials
Proactive Path Provisioning Rate Variance (PPPv) =
SUM[SQUAREOF(PPRi - PPPm)]
-------------------------
Total Trials - 1
Where PPPm is the Average Proactive Path Provisioning Rate.Reporting Format:
Your results MUST be reported in tabular format, with a row for each iteration 📊 The last row of the table is the Proactive Path Provisioning Rate variance, and the row right above it is the Average Proactive Path Provisioning Rate. ✨
Your report should also capture these things, on top of the configuration parameters captured per Section 4.8 💖
- Number of Network Devices in the path
- Offered rate
5.1.8. Network Topology Change Detection Time 🔍
Objective:
- This one clocks how long the controller takes to notice that the network topology changed 🕰️ The timer starts with the notification message the controller receives at its southbound interface and stops with the first topology rediscovery message the controller sends from that same interface. 💅
Reference Test Setup:
- This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🌸
Prerequisites:
- The controller MUST have successfully discovered the network topology information for the deployed network topology. 🫶
- The periodic network discovery operation should be configured to twice the Trial Duration (Td) value ⏱️
Procedure:
- Trigger a topology change event by bringing down an active Network Device in the topology 💥
- Record the time when the first topology change notification is sent to the controller (Tcn) at the forwarding-plane test emulator interface (I1) 📝
- Stop the trial when the controller sends the first topology rediscovery message to the Network Device, or when the Trial Duration (Td) expires 🛑
- Record the time when the first topology rediscovery message is received from the controller (Tcd) at the forwarding-plane test emulator interface (I1) 📝
Measurements:
Network Topology Change Detection Time (TDT1) = Tcd - Tcn
Average Network Topology Change Detection Time =
TDT1 + TDT2 + TDT3 .. TDTn
--------------------------
Total Trials
Network Topology Change Detection Time Variance (NTDv) =
SUM[SQUAREOF(TDTi - NTDm)]
--------------------------
Total Trials - 1
Where NTDm is the Average Network Topology Change
Detection Time.Reporting Format:
The Network Topology Change Detection Time results MUST be reported in tabular format, with a row for each iteration 📊 The last row shows the Network Topology Change Detection Time variance, and the row before it shows the Average Network Topology Change Detection Time. 🎀
5.2. Scalability 📈
5.2.1. Control Sessions Capacity 🧁
Objective:
- Here we measure the maximum number of control sessions the controller can keep alive 🤝 That's the number of sessions it accepts from Network Devices, counting from the first control session to the last one the controller accepts at its southbound interface. ✨
Reference Test Setup:
- This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🫧
Prerequisites:
- None 🙃
Procedure:
- Establish control connections with the controller from every Network Device emulated in the forwarding-plane test emulator 🔌
- Stop the trial when the controller starts dropping the control connections 🛑
- Record the number of successful connections established (CCn) with the controller at the forwarding-plane test emulator 📝
Measurement:
Control Sessions Capacity = CCnReporting Format:
The Control Sessions Capacity results MUST be reported, on top of the configuration parameters captured per Section 4.8 💅
5.2.2. Network Discovery Size 🗺️
Objective:
- This test checks how big a network the controller can discover 🌐 It counts the nodes, links, and hosts. The trial starts with a network topology the user hands over for discovery and ends with the number of nodes, links, and hosts the controller(s) actually managed to discover. 🦋
Reference Test Setup:
- This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🌈
Prerequisites:
- The controller MUST support automatic network discovery. 🤖
- The tester should be able to retrieve the discovered topology information through either the controller's management interface or northbound interface. 👩💻
Procedure:
- Establish the network connections between the controller and the network nodes 🔗
- Query the controller every t seconds (the RECOMMENDED value for t is 30) to get the discovered network topology information through the northbound interface or the management interface ⏰
- Stop the trial when the discovered network topology information stays the same as the last two query responses 🛑
- Compare the obtained network topology information with the deployed network topology information 🔍
- If the comparison is successful, increase the number of nodes by 1 and repeat the trial. If the comparison is unsuccessful, decrease the number of nodes by 1 and repeat the trial. 📈📉
- Keep going until the comparison in step 5 is successful 🏁
- Record the number of nodes for the last trial run (Ns) where the topology comparison was successful 📝
Measurement:
Network Discovery Size = NsReporting Format:
The Network Discovery Size results MUST be reported, on top of the configuration parameters captured per Section 4.8 🎀
5.2.3. Forwarding Table Capacity 🧠
Objective:
- Last one in this bunch! This measures the maximum number of flow entries a controller can manage in its Forwarding Table 💜
Reference Test Setup:
- This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🌸
Prerequisites:
- The controller's Forwarding Table should be empty. 🫙
- "Flow idle time" MUST be set to a higher or infinite value. ♾️
- The controller MUST have successfully completed network topology discovery. 🗺️
- The tester should be able to retrieve the Forwarding Table information through either the controller's management interface or northbound interface. 👩💻
Procedures:
- Reactive Flow Provisioning Mode:
- Send bidirectional traffic continuously with unique source and destination addresses from test traffic generators TP1 and TP2, at the Asynchronous Message Processing Rate of the controller 🔁
- Query the controller at a regular interval (e.g., every 5 seconds) for the number of learned flow entries from its northbound interface ⏱️
- Stop the trial when the retrieved value stays constant for three consecutive iterations, and record the value from the last query (Nrp) 🛑📝
- Proactive Flow Provisioning Mode:
- Install unique flows continuously through the controller's northbound interface or management interface until a failure response comes back from the controller 📦
- Record the total number of successful responses (Nrp) 📝
Note: Some controller designs for Proactive Flow Provisioning mode may require the switch to send flow setup requests in order to generate flow setup responses. In such cases, it is recommended to generate bidirectional traffic for the provisioned flows. 💡
Measurements:
- Proactive Flow Provisioning Mode:
Max Flow Entries = Total number of flows provisioned (Nrp)- Reactive Flow Provisioning Mode:
Max Flow Entries = Total number of learned flow entries (Nrp)- Forwarding Table Capacity = Max Flow Entries 🎯
Reporting Format:
The Forwarding Table Capacity results MUST be tabulated, along with the following information and the configuration parameters captured per Section 4.8 ✨
- Provisioning Type (Proactive/Reactive)
5.3. Security 🔐
5.3.1. Exception Handling 🚨
Objective:
okay so the goal here is to see how handling error packets and notifications changes the results of our performance tests 👀. The impact MUST be measured for the following performance tests:
- Path Provisioning Rate
- Path Provisioning Time
- Network Topology Change Detection Time
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🎀
Prerequisites:
- This test MUST be performed after obtaining the baseline measurement results for the performance tests listed above. 📸
- Ensure that the invalid messages are not dropped by the intermediate devices connecting the controller and Network Devices. 🫶
Procedure:
- Perform the above-listed performance tests, and send 1% of the messages from the Asynchronous Message Processing Rate test (Section 5.1.3) as invalid messages from the connected Network Devices emulated at the forwarding-plane test emulator. 💖
- Perform the above-listed performance tests, and send 2% of the messages from the Asynchronous Message Processing Rate test (Section 5.1.3) as invalid messages from the connected Network Devices emulated at the forwarding-plane test emulator. ✨
Note:
Invalid messages can be frames with incorrect protocol fields or any form of failure notifications sent towards the controller. 🙈
Measurements:
Measurements MUST be done as per the equation defined in the "Measurements" section of the corresponding test listed under "Objective". 📏
Reporting Format:
The Exception Handling results MUST be reported in tabular format, with a column for each of the below parameters and row for each of the above-listed performance tests:
- Without Exceptions
- With 1% Exceptions
- With 2% Exceptions
5.3.2. Handling Denial-of-Service Attacks 🛡️
Objective:
so this one is about how the controller holds up when someone is trying to flood it with DoS attacks 😤. Determine the effects of handling DoS attacks on performance and scalability tests. The impact MUST be measured for the following tests:
- Path Provisioning Rate
- Path Provisioning Time
- Network Topology Change Detection Time
- Network Discovery Size
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🦋
Prerequisite:
This test MUST be performed after obtaining the baseline measurement results for the performance tests listed above. 🧁
Procedure:
Perform the above-listed tests, and launch a DoS attack towards the controller while the trial is running. 💥
Note:
DoS attacks can be launched on one of the following interfaces:
- Northbound (e.g., query for flow entries continuously on the northbound interface)
- Management (e.g., Ping requests to the controller's management interface)
- Southbound (e.g., TCP SYN messages on the southbound interface)
Measurements:
Measurements MUST be done as per the equation defined in the "Measurements" section of the corresponding test listed under "Objective". 📐
Reporting Format:
The results regarding the handling of DoS attacks MUST be reported in tabular format, with a column for each of the below parameters and a row for each of the above-listed tests.
- Without any attacks
- With attacks
The report should also specify the nature of the attack and the interface in question. 🕵️♀️
5.4. Reliability 💜
5.4.1. Controller Failover Time 🔁
Objective:
Measure the time taken to switch from an active controller to the backup controller when the controllers work in redundancy mode and the active controller fails. It's defined as the interval starting when the active controller is brought down and ending with the first rediscovery message received from the new controller at its southbound interface. 🌸
Reference Test Setup:
This test SHOULD use the test setup illustrated in Section 3.2 of this document. 🎀
Prerequisites:
- Master controller election MUST be completed. 👑
- Nodes are connected to the controller cluster per the implemented redundancy mode (e.g., active-standby). 🧩
- The controller cluster should have successfully completed the network topology discovery. 🗺️
- The Network Device MUST send all new flows to the controller when it receives them from the test traffic generator. 🚀
- The controller should have learned the location of the destination (D1) at test traffic generator TP2. 📍
Procedure:
- Send unidirectional traffic continuously with incremental sequence numbers and source addresses from test traffic generator TP1 at the rate at which the controller can process the traffic without any drops. 🌊
- Ensure that there are no packet drops observed at test traffic generator TP2. 👀
- Bring down the active controller. 📉
- Stop the trial when the first frame after the failover operation is received on test traffic generator TP2. 🛑
- Record the time at which the last valid frame was received (T1) at test traffic generator TP2 before the sequence error and the time at which the first valid frame was received (T2) after the sequence error at test traffic generator TP2. ⏱️
Measurements:
Controller Failover Time = (T2 - T1)
Packet Loss = Number of missing packet sequencesReporting Format:
The Controller Failover Time results MUST be tabulated with the following information:
- Number of cluster nodes
- Redundancy mode
- Controller Failover Time
- Packet Loss
- Cluster keep-alive interval
5.4.2. Network Re-provisioning Time 🛤️
Objective:
Measure the time taken by the controller to reroute traffic when there is a failure in existing traffic paths. It's defined as the interval starting with the first failure notification message received by the controller and ending with the last flow re-provisioning message sent by the controller at its southbound interface. 🦋
Reference Test Setup:
This test SHOULD use one of the test setups illustrated in Section 3.1 or Section 3.2 of this document. 🧚
Prerequisites:
- A network with a specified number of nodes and redundant paths MUST be deployed. 🕸️
- The controller MUST know the location of test traffic generators TP1 and TP2. 🧭
- Ensure that the controller does not pre-provision the alternate path in the emulated Network Devices at the forwarding-plane test emulator. 🙅♀️
Procedure:
- Send bidirectional traffic continuously with a unique sequence number from test traffic generators TP1 and TP2. ↔️
- Bring down a link or switch in the traffic path. 💔
- Stop the trial after receiving the first frame after network reconvergence. 🛑
- Record the time of the last received frame prior to the frame loss at test traffic generator TP2 (TP2-Tlfr) and the time of the first frame received after the frame loss at test traffic generator TP2 (TP2-Tffr). There must be a gap in sequence numbers of these frames. 🔍
- Record the time of the last received frame prior to the frame loss at test traffic generator TP1 (TP1-Tlfr) and the time of the first frame received after the frame loss at test traffic generator TP1 (TP1-Tffr). 📝
Measurements:
Forward Direction Path Re-provisioning Time (FDRT)
= (TP2-Tffr - TP2-Tlfr)
Reverse Direction Path Re-provisioning Time (RDRT)
= (TP1-Tffr - TP1-Tlfr)
Network Re-provisioning Time = (FDRT + RDRT)/2
Forward Direction Packet Loss = Number of missing sequence frames
at test traffic generator TP1
Reverse Direction Packet Loss = Number of missing sequence frames
at test traffic generator TP2Reporting Format:
The Network Re-provisioning Time results MUST be tabulated with the following information:
- Number of nodes in the primary path
- Number of nodes in the alternate path
- Network Re-provisioning Time
- Forward Direction Packet Loss
- Reverse Direction Packet Loss
6. IANA Considerations 🌈
This document has no IANA actions. 🫶
7. Security Considerations 🔐
okay this section is serious, bestie, so read it carefully 🚨
The benchmarking tests described in this document are limited to the performance characterization of controllers in a lab environment with isolated networks. 🧪
The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network or misroute traffic to the test management network. 🚫
Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the controller. 📦
Special capabilities SHOULD NOT exist in the controller specifically for benchmarking purposes. Any implications for network security arising from the controller SHOULD be identical in the lab and in production networks. 🔒
8. References 📚
8.1. Normative References
[RFC2119]Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.[RFC8174]Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.[RFC8455]Bhuvaneswaran, V., Basil, A., Tassinari, M., Manral, V., and S. Banks, "Terminology for Benchmarking Software-Defined Networking (SDN) Controller Performance", RFC 8455, DOI 10.17487/RFC8455, October 2018, <https://www.rfc-editor.org/info/rfc8455>.
8.2. Informative References
[OpenFlow-Spec]ONF, "OpenFlow Switch Specification" Version 1.4.0 (Wire Protocol 0x05), October 2013, <https://www.opennetworking.org/wp-content/uploads/2014/10/openflow-spec-v1.4.0.pdf>.
Appendix A. Benchmarking Methodology Using OpenFlow Controllers 🦋
This section gives an overview of the OpenFlow protocol [OpenFlow-Spec] and provides a test methodology for benchmarking SDN Controllers supporting the OpenFlow southbound protocol. The OpenFlow protocol is used as an example to illustrate the methodologies defined in this document. ✨
A.1. Protocol Overview 🧠
OpenFlow [OpenFlow-Spec] is an open standard protocol defined by the Open Networking Foundation (ONF) and used for programming the forwarding plane of network switches or routers via a centralized controller. 💅
A.2. Messages Overview ✨
okay so OpenFlow has three message types 💅: controller-to-switch, asynchronous, and symmetric. let's break them down, bestie 🫶
Controller-to-switch messages are started by the controller, and they're how it directly manages or peeks at the switch 👀. Controllers use them to query and configure the switch (features messages, configuration messages), collect info from a switch (Read-State messages), send packets out a specific port of a switch (OFPT_PACKET_OUT messages), and change the switch's forwarding plane and state (Modify-State messages, Role-Request messages, etc.). it's giving total control 👑
Asynchronous messages are generated by the switch without the controller asking for them 🧚. They let switches update controllers when a new flow shows up (OFPT_PACKET_IN messages), when switch state changes ("flow-removed" messages, port-status messages), and when something goes wrong (Error messages). the switch literally just speaks up when it needs to 📣
Symmetric messages can be generated in either direction without anyone asking 🦋. They let switches and controllers set up a connection (Hello messages), check that the other side is still alive (Echo messages), and offer extra features (Experimenter messages). lowkey the chill ones 🌸
A.3. Connection Overview
okay so the OpenFlow channel is where OpenFlow messages get swapped between an OpenFlow switch and an OpenFlow controller 🎀. The channel can be set up using plain TCP or TLS. by default, a switch makes a single connection with the SDN Controller 💜. but a switch may also set up multiple parallel connections to a single controller (auxiliary connection), or connect to multiple controllers to handle controller failures and load balancing ✨. it's the ultimate backup plan energy 🫶
A.4. Performance Benchmarking Tests 🍓
A.4.1. Network Topology Discovery Time 🌈
Procedure:
Network Devices OpenFlow SDN
Controller Application
| | |
| |<Initialize controller |
| |app., NB and SB interfaces>|
| | |
|<Deploy network with | |
| given no. of OF switches> | |
| | |
| OFPT_HELLO Exchange | |
|<-------------------------->| |
| | |
| OFPT_PACKET_OUT with LLDP| |
| to all switches| |
(Tm1)|<---------------------------| |
| | |
| OFPT_PACKET_IN with LLDP| |
| rcvd from Switch 1| |
|--------------------------->| |
| | |
| OFPT_PACKET_IN with LLDP| |
| rcvd from Switch 2| |
|--------------------------->| |
| . | |
| . | |
| | |
| OFPT_PACKET_IN with LLDP| |
| rcvd from Switch n| |
(Tmn)|--------------------------->| |
| | |
| | <Wait for the expiry of|
| | the Trial Duration (Td)>|
| | |
| | Query the controller for|
| | discovered n/w topo. (Di)|
| |<--------------------------|
| | |
| | <Compare the discovered|
| | n/w topology and the|
| | offered n/w topology>|
| | |Legend:
- NB: Northbound
- SB: Southbound
- OF: OpenFlow
- OFP: OpenFlow Protocol
- LLDP: Link-Layer Discovery Protocol
- Tm1: Time of reception of first LLDP message from controller
- Tmn: Time of last LLDP message sent to controller
Discussion:
ok the Network Topology Discovery Time is the gap between the first OFPT_PACKET_OUT carrying an LLDP message received from the controller (Tm1) and the last OFPT_PACKET_IN carrying an LLDP message sent to the controller (Tmn), as long as the comparison works out ✅. it's giving precision 🔥
A.4.2. Asynchronous Message Processing Time 💖
Procedure:
Network Devices OpenFlow SDN
Controller Application
| | |
|OFPT_PACKET_IN with single | |
|OFP match header | |
(T0)|--------------------------->| |
| | |
|OFPT_PACKET_OUT with single | |
| OFP action header | |
(R0)|<---------------------------| |
| . | |
| . | |
| . | |
| | |
|OFPT_PACKET_IN with single | |
|OFP match header | |
(Tn)|--------------------------->| |
| | |
|OFPT_PACKET_OUT with single | |
| OFP action header | |
(Rn)|<---------------------------| |
| | |
|<Wait for the expiry of the | |
|Trial Duration> | |
| | |
|<Record the number of | |
|OFPT_PACKET_INs/ | |
|OFPT_PACKET_OUTs | |
|exchanged (Nrx)> | |
| | |Legend:
- T0,T1, ..Tn: transmit timestamps of
OFPT_PACKET_INmessages - R0,R1, ..Rn: receive timestamps of
OFPT_PACKET_OUTmessages - Nrx: Number of successful
OFPT_PACKET_IN/OFPT_PACKET_OUTmessage exchanges
Discussion:
the Asynchronous Message Processing Time comes from adding up the differences ((R0 - T0),(R1 - T1)..(Rn - Tn)) and then dividing that sum by Nrx 🧮. the math is so clean, I'm obsessed 😭
A.4.3. Asynchronous Message Processing Rate 🎀
Procedure:
Network Devices OpenFlow SDN
Controller Application
| | |
|OFPT_PACKET_IN with single | |
|OFP match header | |
|--------------------------->| |
| | |
|OFPT_PACKET_OUT with single | |
| OFP action header | |
|<---------------------------| |
| | |
| . | |
| . | |
| . | |
| | |
|OFPT_PACKET_IN with single | |
|OFP match header | |
|--------------------------->| |
| | |
|OFPT_PACKET_OUT with single | |
| OFP action header | |
|<---------------------------| |
| | |
|<Repeat the steps until | |
|the expiry of the | |
|Trial Duration> | |
| | |
|<Record the number of OFP | |
(Ntx1)|match headers sent> | |
| | |
|<Record the number of OFP | |
(Nrx1)|action headers rcvd> | |
| | |Note: The Ntx1 on initial trials should be greater than Nrx1. Repeat the trials until the Nrxn for two consecutive trials equals (+/-P%).
Discussion:
this one is a single procedure that measures two benchmarks 🔥
- The Maximum Asynchronous Message Processing Rate comes from finding the maximum
OFPT_PACKET_OUTs (Nrxn) received from the controller(s) across n trials 👑. - The Loss-Free Asynchronous Message Processing Rate comes from finding the maximum
OFPT_PACKET_OUTs received from the controller(s) when the Loss Ratio equals zero 🌸. The Loss Ratio is calculated as1 - Nrxn/Ntxn🙈.
two benchmarks, one procedure, zero notes-app chaos ✨🌈
A.4.4. Reactive Path Provisioning Time
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow
Generator TP1 Generator TP2 Controller
| | | |
| |G-ARP (D1) | |
| |--------------------->| |
| | | |
| | |OFPT_PACKET_IN(D1) |
| | |-------------------->|
| | | |
|Traffic (S1,D1) | |
(Tsf1)|----------------------------------->| |
| | | |
| | | |
| | | |
| | |OFPT_PACKET_IN(S1,D1)|
| | |-------------------->|
| | | |
| | | FLOW_MOD(D1) |
| | |<--------------------|
| | | |
| |Traffic (S1,D1) | |
| (Tdf1)|<---------------------| |
| | | |Legend:
G-ARP: Gratuitous ARP message
Tsf1: Time of first frame sent from TP1
Tdf1: Time of first frame received from TP2Discussion:
Okay bestie, so this one is the reactive path provisioning time 🦋 The vibe is: the switches don't know where to send your traffic yet, so they ask the controller (👑 the OpenFlow Controller) and wait for it to install a flow. The Reactive Path Provisioning Time is just the gap between when the first frame leaves TP1 and when the first frame shows up at TP2, aka Tsf1 - Tdf1 💅 Literally that's it, you subtract the times and you're done, no cap ✨
A.4.5. Proactive Path Provisioning Time
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application
| | | | |
| | | | |
| | | |<Install flow|
| | | | for S1,D1> |
| |G-ARP (D1) | | |
| |-------------->| | |
| | | | |
| | |OFPT_PACKET_IN(D1)| |
| | |----------------->| |
| | | | |
|Traffic (S1,D1) | | |
(Tsf1)|--------------------------->| | |
| | | | |
| | | FLOW_MOD(D1) | |
| | |<-----------------| |
| | | | |
| |Traffic (S1,D1)| | |
| (Tdf1)|<--------------| | |
| | | | |Legend:
G-ARP: Gratuitous ARP message
Tsf1: Time of first frame sent from TP1
Tdf1: Time of first frame received from TP2Discussion:
Okay so the proactive version is giving "I planned ahead" energy 🌸 Here the SDN Application installs the flow for S1,D1 before the traffic even starts, so the controller already has the answer when things kick off. The Proactive Path Provisioning Time is found the same way: the difference between the transmit and receive times of the traffic, Tsf1 - Tdf1 🎀 Same math, cuter setup, obsessed 😭
A.4.6. Reactive Path Provisioning Rate
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow
Generator TP1 Generator TP2 Controller
| | | |
| | | |
| | | |
| |G-ARP (D1..Dn) | |
| |--------------------| |
| | | |
| | |OFPT_PACKET_IN(D1..Dn)|
| | |--------------------->|
| | | |
|Traffic (S1..Sn,D1..Dn) | |
|--------------------------------->| |
| | | |
| | |OFPT_PACKET_IN(S1..Sn,|
| | | D1..Dn)|
| | |--------------------->|
| | | |
| | | FLOW_MOD(S1) |
| | |<---------------------|
| | | |
| | | FLOW_MOD(D1) |
| | |<---------------------|
| | | |
| | | FLOW_MOD(S2) |
| | |<---------------------|
| | | |
| | | FLOW_MOD(D2) |
| | |<---------------------|
| | | . |
| | | . |
| | | |
| | | FLOW_MOD(Sn) |
| | |<---------------------|
| | | |
| | | FLOW_MOD(Dn) |
| | |<---------------------|
| | | |
| | Traffic (S1..Sn, | |
| | D1..Dn)| |
| |<-------------------| |
| | | |
| | | |Legend:
G-ARP: Gratuitous ARP message
D1..Dn: Destination Endpoint 1, Destination Endpoint 2 ...,
Destination Endpoint n
S1..Sn: Source Endpoint 1, Source Endpoint 2 ...,
Source Endpoint nDiscussion:
Okay now we're going big 👑 This is the reactive rate, so instead of one flow we're setting up a whole bunch of them (D1..Dn and S1..Sn) and the controller is working through all of them 💖 The Reactive Path Provisioning Rate comes from counting the total number of frames that land at test traffic generator TP2 after the Trial Duration. More frames, more slay 🔥
A.4.7. Proactive Path Provisioning Rate
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application
| | | | |
| |G-ARP (D1..Dn) | | |
| |--------------->| | |
| | | | |
| | |OFPT_PACKET_IN | |
| | | (D1..Dn)| |
| | |---------------->| |
| | | | |
|Traffic (S1..Sn,D1..Dn) | | |
(Tsf1)|---------------------------->| | |
| | | | |
| | | |<Install flow|
| | | | for S1,D1> |
| | | | |
| | | | . |
| | | |<Install flow|
| | | | for Sn,Dn> |
| | | | |
| | | FLOW_MOD(S1) | |
| | |<----------------| |
| | | | |
| | | FLOW_MOD(D1) | |
| | |<----------------| |
| | | | |
| | | . | |
| | | FLOW_MOD(Sn) | |
| | |<----------------| |
| | | | |
| | | FLOW_MOD(Dn) | |
| | |<----------------| |
| | | | |
| |Traffic (S1..Sn,| | |
| | D1..Dn)| | |
| (Tdf1)|<---------------| | |
| | | | |Legend:
G-ARP: Gratuitous ARP message
D1..Dn: Destination Endpoint 1, Destination Endpoint 2 ...,
Destination Endpoint n
S1..Sn: Source Endpoint 1, Source Endpoint 2 ...,
Source Endpoint nDiscussion:
Last but not least, the proactive rate 🌈 Same big multi-endpoint energy as before, except the SDN Application has already installed the flows for S1,D1 through Sn,Dn ahead of time, so the controller is prepped and ready 🧚 The Proactive Path Provisioning Rate is found by counting the total number of frames received at test traffic generator TP2 after the Trial Duration, just like the reactive one. Iconic behavior, periodt 💜
A.4.8. Network Topology Change Detection Time ✨
Procedure:
Network Devices OpenFlow SDN
Controller Application
| | |
| | <Bring down a link in |
| | Switch S1>|
| | |
T0 |PORT_STATUS with link down | |
| from S1 | |
|--------------------------->| |
| | |
|First OFPT_PACKET_OUT with | |
|LLDP to OF switch | |
T1 |<---------------------------| |
| | |
| | <Record time of first|
| | OFPT_PACKET_OUT with|
| | LLDP T1>|
| | |Discussion:
Okay, so this one is about how fast the network notices a link went down! 💖 You get it by subtracting two timestamps: T0, the moment OpenFlow Switch S1 sends its PORT_STATUS message, and T1, the moment the OpenFlow controller sends its first topology rediscovery message to the OpenFlow switches. The gap between them is the Network Topology Change Detection Time. It's giving precision, bestie 🧚
A.5. Scalability 🦋
A.5.1. Control Sessions Capacity 💅
Procedure:
Network Devices OpenFlow
Controller
| |
| OFPT_HELLO Exchange for Switch 1 |
|<------------------------------------->|
| |
| OFPT_HELLO Exchange for Switch 2 |
|<------------------------------------->|
| . |
| . |
| . |
| OFPT_HELLO Exchange for Switch n |
|X<----------------------------------->X|
| |Discussion:
Lowkey the cutest test ever 🎀 The value of Switch (n - 1) gives you the Control Sessions Capacity. So the controller does a HELLO handshake with each switch one by one, and the last one that works is the number that matters 👑
A.5.2. Network Discovery Size 🌸
Procedure:
Network Devices OpenFlow SDN
Controller Application
| | |
| | <Deploy network with |
| |given no. of OF switches N>|
| | |
| OFPT_HELLO Exchange | |
|<-------------------------->| |
| | |
| OFPT_PACKET_OUT with LLDP| |
| to all switches | |
|<---------------------------| |
| | |
| OFPT_PACKET_IN with LLDP| |
| rcvd from Switch 1| |
|--------------------------->| |
| | |
| OFPT_PACKET_IN with LLDP| |
| rcvd from Switch 2| |
|--------------------------->| |
| . | |
| . | |
| | |
| OFPT_PACKET_IN with LLDP| |
| rcvd from Switch n| |
|--------------------------->| |
| | |
| | <Wait for the expiry of|
| | the Trial Duration (Td)>|
| | |
| | Query the controller for|
| | discovered n/w topo. (N1)|
| |<--------------------------|
| | |
| | <If N1==N, repeat Step 1|
| | with N + 1 nodes|
| | until N1<N >|
| | |
| | <If N1<N, repeat Step 1 |
| | with N=N1 nodes once and |
| | exit> |
| | |Legend:
n/w topo: Network topologyOF: OpenFlow
Discussion:
Okay, the vibe is "how big can the network get before the controller loses track" 🧚 The value of N1 gives you the Network Discovery Size. The Trial Duration can be set to whatever time the user expects the controller to finish discovery in. Main character energy for the controller, no cap 💅
A.5.3. Forwarding Table Capacity 💜
Procedure:
Test Traffic Network Devices OpenFlow SDN
Generator TP1 Controller Application
| | | |
| | | |
|G-ARP (H1..Hn) | | |
|---------------->| | |
| | | |
| |OFPT_PACKET_IN(D1..Dn)| |
| |--------------------->| |
| | | |
| | |<Wait for 5 secs>|
| | | |
| | | <Query for FWD |
| | | entry> |(F1)
| | | |
| | |<Wait for 5 secs>|
| | | |
| | | <Query for FWD |
| | | entry> |(F2)
| | | |
| | |<Wait for 5 secs>|
| | | |
| | | <Query for FWD |
| | | entry> |(F3)
| | | |
| | | <Repeat Step 2 |
| | |until F1==F2==F3>|
| | | |Legend:
G-ARP: Gratuitous ARP messageH1..Hn: Host 1 .. Host nFWD: Forwarding Table
Discussion:
Okay bestie, this one is a little bit of a stalking situation 🙈 The controller's Forwarding Table entries get queried over and over until three consecutive queries give back the same value. Whatever value came back last is your Forwarding Table Capacity. The query interval is totally up to you to configure, and the 5 seconds in this example is only there to make it easier to picture 🌈
A.6. Security 🔥
A.6.1. Exception Handling 🫶
Procedure: okay bestie, here's the setup 🧪✨
Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application
| | | | |
| |G-ARP (D1..Dn) | | |
| |--------------->| | |
| | | | |
| | |OFPT_PACKET_IN(D1..Dn)| |
| | |--------------------->| |
| | | | |
|Traffic (S1..Sn,D1..Dn) | | |
|-------------------------->| | |
| | | | |
| | |OFPT_PACKET_IN(S1..Sa,| |
| | | D1..Da)| |
| | |--------------------->| |
| | | | |
| | |OFPT_PACKET_IN | |
| | | (Sa+1..Sn,| |
| | | Da+1..Dn)| |
| | | (1% incorrect OFP| |
| | | match header)| |
| | |--------------------->| |
| | | | |
| | | FLOW_MOD(D1..Dn)| |
| | |<---------------------| |
| | | | |
| | | FLOW_MOD(S1..Sa)| |
| | | OFP headers| |
| | |<---------------------| |
| | | | |
| |Traffic (S1..Sa,| | |
| | D1..Da)| | |
| |<---------------| | |
| | | | |
| | | | <Wait for the|
| | | | expiry of the|
| | | | Trial|
| | | | Duration>|
| | | | |
| | | | <Record Rx|
| | | | frames at|
| | | | TP2 (Rn1)>|
| | | | |
| | | | <Repeat |
| | | | Step 1 with|
| | | | 2% incorrect|
| | | |OFPT_PACKET_INs>|
| | | | |
| | | | <Record Rx|
| | | | frames at|
| | | | TP2 (Rn2)>|Legend: here's the decoder ring 💅
G-ARP: Gratuitous ARP messageOFPT_PACKET_IN(Sa+1..Sn,Da+1..Dn): OFPT_PACKET_IN with the wrong version number 🙈Rn1: the total number of frames received at Test Port 2 when 1% of the frames are incorrectRn2: the total number of frames received at Test Port 2 when 2% of the frames are incorrect
Discussion: this is where it gets spicy 🔥
The traffic rate sent towards the OpenFlow switch from Test Port 1 should be 1% higher than the Path Programming Rate. Rn1 gives you the controller's Path Provisioning Rate when 1% of the frames are incorrect, and Rn2 gives you the Path Provisioning Rate when 2% are incorrect. Main character energy for the controller, honestly 👑
The procedure above gives you the test steps to figure out how handling error packets affects the Path Programming Rate. And the same procedure can be adapted to check the effects on the other performance tests in this benchmarking test. Iconic reuse 🦋
A.6.2. Handling Denial-of-Service Attacks ✨
Procedure: the DoS era 😭🌈
Test Traffic Test Traffic Network Device OpenFlow SDN
Generator TP1 Generator TP2 Controller Application
| | | | |
| |G-ARP (D1..Dn) | | |
| |---------------->| | |
| | | | |
| | |OFPT_PACKET_IN(D1..Dn)| |
| | |--------------------->| |
| | | | |
|Traffic (S1..Sn,D1..Dn) | | |
|--------------------------->| | |
| | | | |
| | |OFPT_PACKET_IN(S1..Sn,| |
| | | D1..Dn)| |
| | |--------------------->| |
| | | | |
| | |TCP SYN attack | |
| | |from a switch | |
| | |--------------------->| |
| | | | |
| | |FLOW_MOD(D1..Dn) | |
| | |<---------------------| |
| | | | |
| | | FLOW_MOD(S1..Sn) | |
| | | OFP headers | |
| | |<---------------------| |
| | | | |
| |Traffic (S1..Sn, | | |
| | D1..Dn) | | |
| |<----------------| | |
| | | | |
| | | |<Wait for the|
| | | |expiry of the|
| | | | Trial|
| | | | Duration>|
| | | | |
| | | | <Record Rx|
| | | | frames at|
| | | | TP2 (Rn1)>|
| | | | |Legend: just the one term this time 🍓
G-ARP: Gratuitous ARP message
Discussion: a little chaos, a lot of learning 💀
A TCP SYN attack should be launched from one of the emulated or simulated OpenFlow switches. Rn1 tells you the controller's Path Programming Rate while it's handling the denial-of-service attack. Stressful, but it's giving science 🧪
The procedure above gives you the test steps to see how handling denial of service affects the Path Programming Rate. The same procedure can be adapted to check the effects on the other performance tests in this benchmarking test. Bestie, you're literally building a whole toolkit 🫶
A.7. Reliability 🌸
A.7.1. Controller Failover Time 🦋
Okay bestie, the first diagram picks up right where we left off 💖
Procedure:
Test Traffic Test Traffic Network Device OpenFlow SDN
Generator TP1 Generator TP2 Controller Application
| | | | |
| |G-ARP (D1) | | |
| |-------------->| | |
| | | | |
| | |OFPT_PACKET_IN(D1) | |
| | |---------------------->| |
| | | | |
|Traffic (S1..Sn,D1) | | |
|--------------------------->| | |
| | | | |
| | | | |
| | |OFPT_PACKET_IN(S1,D1) | |
| | |---------------------->| |
| | | | |
| | |FLOW_MOD(D1) | |
| | |<----------------------| |
| | |FLOW_MOD(S1) | |
| | |<----------------------| |
| | | | |
| |Traffic (S1,D1)| | |
| |<--------------| | |
| | | | |
| | |OFPT_PACKET_IN(S2,D1) | |
| | |---------------------->| |
| | | | |
| | |FLOW_MOD(S2) | |
| | |<----------------------| |
| | | | |
| | |OFPT_PACKET_IN | |
| | | (Sn-1,D1) | |
| | |---------------------->| |
| | | | |
| | |OFPT_PACKET_IN(Sn,D1) | |
| | |---------------------->| |
| | | . | |
| | | . |<Bring down |
| | | . | the active |
| | | | controller> |
| | | FLOW_MOD(Sn-1) | |
| | | X<-----------------| |
| | | | |
| | |FLOW_MOD(Sn) | |
| | |<----------------------| |
| | | | |
| |Traffic (Sn,D1)| | |
| |<--------------| | |
| | | | |
| | | |<Stop the |
| | | |test after |
| | | |recv. traffic|
| | | |upon |
| | | |failure> |Legend:
G-ARP: Gratuitous ARP message
Discussion:
Okay so the gap between the last valid frame you receive before the traffic loss and the first frame you receive after it = the Controller Failover Time 🦋✨ That's the number we're after!
If there's no frame loss during the Controller Failover Time, it can be deemed negligible 💅🌸 (no drama, no problem, iconic).
A.7.2. Network Re-provisioning Time 👑
Procedure:
Test Traffic Test Traffic Network Devices OpenFlow SDN
Generator TP1 Generator TP2 Controller Application
| | | | |
| |G-ARP (D1) | | |
| |--------------->| | |
| | | | |
| | |OFPT_PACKET_IN(D1) | |
| | |--------------------->| |
| |G-ARP (S1) | | |
|----------------------------->| | |
| | | | |
| | |OFPT_PACKET_IN(S1) | |
| | |--------------------->| |
| | | | |
|Traffic (S1,D1,Seq. no (1..n))| | |
|----------------------------->| | |
| | | | |
| | |OFPT_PACKET_IN(S1,D1) | |
| | |--------------------->| |
| | | | |
| | Traffic (D1,S1,| | |
| | Seq. no (1..n))| | |
| |--------------->| | |
| | | | |
| | |OFPT_PACKET_IN(D1,S1) | |
| | |--------------------->| |
| | | | |
| | |FLOW_MOD(D1) | |
| | |<---------------------| |
| | | | |
| | |FLOW_MOD(S1) | |
| | |<---------------------| |
| | | | |
| | Traffic (S1,D1,| | |
| | Seq. no(1))| | |
| |<---------------| | |
| | | | |
| | Traffic (S1,D1,| | |
| | Seq. no(2))| | |
| |<---------------| | |
| | | | |
| | | | |
| Traffic (D1,S1,Seq. no(1))| | |
|<-----------------------------| | |
| | | | |
| Traffic (D1,S1,Seq. no(2))| | |
|<-----------------------------| | |
| | | | |
| Traffic (D1,S1,Seq. no(x))| | |
|<-----------------------------| | |
| | | | |
| | Traffic (S1,D1,| | |
| | Seq. no(x))| | |
| |<---------------| | |
| | | | |
| | | | |
| | | | <Bring down |
| | | | the switch in|
| | | | the active|
| | | | traffic path>|
| | | | |
| | |PORT_STATUS(Sa) | |
| | |--------------------->| |
| | | | |
| | Traffic (S1,D1,| | |
| | Seq. no(n - 1))| | |
| | X<------------| | |
| | | | |
|Traffic (D1,S1,Seq. no(n - 1))| | |
| X<------------------------| | |
| | | | |
| | | | |
| | |FLOW_MOD(D1) | |
| | |<---------------------| |
| | | | |
| | |FLOW_MOD(S1) | |
| | |<---------------------| |
| | | | |
| Traffic (D1,S1,Seq. no(n))| | |
|<-----------------------------| | |
| | | | |
| | Traffic (S1,D1,| | |
| | Seq. no(n))| | |
| |<---------------| | |
| | | | |
| | | |<Stop the test|
| | | | after recv. |
| | | | traffic upon|
| | | | failover> |Legend:
G-ARP: Gratuitous ARP message Seq. no: Sequence number Sa: Neighbor switch of the switch that was brought down
Discussion:
So here's the vibe: the time difference between the last valid frame received before the traffic loss (packet with sequence number x) and the first frame received after the traffic loss (packet with sequence number n) = the Network Re-provisioning Time 🌈💜
⚠️ Bestie, this one's picky! The trial is valid only when the controller provisions the alternate path upon network failure. No controller doing the work = no valid trial 🙈
Acknowledgments ✨
The authors would like to thank the following individuals for providing their valuable comments regarding the earlier draft versions of this document: Al Morton (AT&T), Sandeep Gangadharan (HP), M. Georgescu (NAIST), Andrew McGregor (Google), Scott Bradner, Jay Karthik (Cisco), Ramki Krishnan (VMware), Boris Khasanov (Huawei), and Brian Castelli (Spirent) 🫶💖
Authors' Addresses ✨
Okay bestie, time to meet the squad behind this RFC 💖 Here are the five authors and where you can find them 🦋
Bhuvaneswaran Vengainathan 🌸 Veryx Technologies Inc. 1 International Plaza, Suite 550 Philadelphia, PA 19113 United States of America
Email: bhuvaneswaran.vengainathan@veryxtech.com
Anton Basil 🎀 Veryx Technologies Inc. 1 International Plaza, Suite 550 Philadelphia, PA 19113 United States of America
Email: anton.basil@veryxtech.com
Mark Tassinari 🧚 Hewlett Packard Enterprise 8000 Foothills Blvd. Roseville, CA 95747 United States of America
Email: mark.tassinari@hpe.com
Vishwas Manral 💜 NanoSec Co 3350 Thomas Rd. Santa Clara, CA 95054 United States of America
Email: vishwas.manral@gmail.com
Sarah Banks 👑 VSS Monitoring 930 De Guigne Drive Sunnyvale, CA 94085 United States of America
Email: sbanks@encrypted.net
That's the whole RFC, bestie! 🌈✨ Five authors, one spec, zero cap 💅
📦 where this page comes from
- ietf: rfc8456 2018-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms