RFC 7501

Terminology for Benchmarking Session Initiation Protocol (SIP) Devices: Basic Session Setup and Registration

📅 published April 2015, from ietf

RFC 7501 🎀 Title Block

okay bestie, let's get into RFC 7501 💖 it's an IETF doc from April 2015, and here's the key stuff up top ✨

  • Request for Comments: 7501
  • Category: Informational (this one is published for information, not as a standards track spec 📖)
  • ISSN: 2070-1721
  • Date: April 2015
  • Authors: C. Davids (Illinois Institute of Technology), V. Gurbani (Bell Laboratories, Alcatel-Lucent), and S. Poretsky (Allot Communications) 🫶
  • Obsoletes and updates: the header doesn't list anything it obsoletes or updates 🙈

Terminology for Benchmarking Session Initiation Protocol (SIP) Devices: Basic Session Setup and Registration 🦋

Abstract 🌸

okay so this doc gives you a terminology for benchmarking how well devices perform on SIP ✨ the methodology lives in the companion doc, RFC 7502, and together they let you get benchmarks and compare different kinds of devices like SIP Proxy Servers, Registrars, and Session Border Controllers 💅

when it says "performance" here, it means how much SIP messaging the Device Under Test (DUT) can handle. media streams only show up to see how they affect the signaling behavior 🎧 the goal is a normalized set of tests so you can objectively compare SIP device capacity, which is literally so important bc SIP allows a wild range of configurations and operating conditions that can mess with the numbers 😭 so you need test setup parameters and a methodology, and a standard terminology makes sure benchmarks mean the same thing and were measured the same way 👑

Status of This Memo 📜

not gonna lie, this part is a bit legal-y, but it's important 🙈 this is not an Internet Standards Track specification. it's published for informational purposes only 💜

it's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community 🌈 it got public review and was approved for publication by the Internet Engineering Steering Group (IESG). not every IESG-approved doc is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 for that ✨

for info on the current status, any errata, and how to send feedback, head to http://www.rfc-editor.org/info/rfc7501 🦋

Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. 💖

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info), as they stood on the publication date. please read those carefully bc they spell out your rights and restrictions for this doc 📖 and if you pull Code Components out of this doc, they have to include the Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License says 🫶

Table of Contents 🗺️

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
  1.1.  Scope . . . . . . . . . . . . . . . . . . . . . . . . . .   5
2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   6
3.  Term Definitions  . . . . . . . . . . . . . . . . . . . . . .   7
  3.1.  Protocol Components . . . . . . . . . . . . . . . . . . .   7
    3.1.1.  Session . . . . . . . . . . . . . . . . . . . . . . .   7
    3.1.2.  Signaling Plane . . . . . . . . . . . . . . . . . . .   8
    3.1.3.  Media Plane . . . . . . . . . . . . . . . . . . . . .   8
    3.1.4.  Associated Media  . . . . . . . . . . . . . . . . . .   9
    3.1.5.  Overload  . . . . . . . . . . . . . . . . . . . . . .   9
    3.1.6.  Session Attempt . . . . . . . . . . . . . . . . . . .  10
    3.1.7.  Established Session . . . . . . . . . . . . . . . . .  10
    3.1.8.  Session Attempt Failure . . . . . . . . . . . . . . .  11
  3.2.  Test Components . . . . . . . . . . . . . . . . . . . . .  11
    3.2.1.  Emulated Agent  . . . . . . . . . . . . . . . . . . .  11
    3.2.2.  Signaling Server  . . . . . . . . . . . . . . . . . .  12
    3.2.3.  SIP Transport Protocol  . . . . . . . . . . . . . . .  12
  3.3.  Test Setup Parameters . . . . . . . . . . . . . . . . . .  13
    3.3.1.  Session Attempt Rate  . . . . . . . . . . . . . . . .  13
    3.3.2.  Establishment Threshold Time  . . . . . . . . . . . .  13
    3.3.3.  Session Duration  . . . . . . . . . . . . . . . . . .  14
    3.3.4.  Media Packet Size . . . . . . . . . . . . . . . . . .  14
    3.3.5.  Codec Type  . . . . . . . . . . . . . . . . . . . . .  15
  3.4.  Benchmarks  . . . . . . . . . . . . . . . . . . . . . . .  15
    3.4.1.  Session Establishment Rate  . . . . . . . . . . . . .  15
    3.4.2.  Registration Rate . . . . . . . . . . . . . . . . . .  16
    3.4.3.  Registration Attempt Rate . . . . . . . . . . . . . .  17
4.  Security Considerations . . . . . . . . . . . . . . . . . . .  17
5.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  18
  5.1.  Normative References  . . . . . . . . . . . . . . . . . .  18
  5.2.  Informative References  . . . . . . . . . . . . . . . . .  18
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  19
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  20

1. Introduction 🌈

okay so Service Providers and IT orgs deliver Voice Over IP (VoIP) and multimedia services built on the IETF Session Initiation Protocol (SIP) [RFC3261] 💅 SIP was originally made to set up, tear down, and change streams of media between end users, but it's evolved so much that it now shows up in a growing pile of services and apps 🦋 a lot of those create a media session, but some don't, like text messaging and subscription services 💬

the benchmarking terms in this doc are meant for any SIP-enabled device doing SIP functions inside the network, whether or not it creates media sessions ✨ performance of end-user devices is out of scope, so don't come for the phones here 🙈

a bunch of networking devices exist to support SIP-based VoIP, including SIP servers, Session Border Controllers (SBCs), and Back-to-back User Agents (B2BUAs) 🔥 these mix voice and IP functions, and their performance might be reported with metrics the manufacturer or vendor picked 🌸 but if a Service Provider or IT org wants to compare devices, they can't do it with those vendor-specific metrics, bc the test conditions and collection algorithms are often unspecified 😭

SIP functional elements and the devices that contain them can be configured in lots of ways and arranged in different topologies 🧚 those choices change the value of any signaling benchmark you pick 💜 unless the conditions of test are defined, you literally can't do a true comparison of performance metrics across multiple vendor implementations 💀

some SIP-enabled devices terminate or relay media as well as signaling, and how a device processes media changes its signaling performance ✨ so the conditions of test must include whether or not the DUT processes media 🎀 and if the device processes media during the test, a description of that media must be provided 📝 this doc and its companion methodology doc [RFC7502] give you a set of black-box benchmarks for describing and comparing devices that have the SIP User Agent Client and Server functions and operate in the network's core 👑

defining SIP performance benchmarks necessarily includes defining Test Setup Parameters and a test methodology 🫶 these let the Tester benchmark different devices and get comparable results 🦋 this doc gives you a shared set of definitions for Test Components, Test Setup Parameters, and Benchmarks 💖 every benchmark defined here is a black-box measurement of the SIP signaling plane 🔥 and the Test Setup Parameters and Benchmarks are meant to be used together with the companion methodology doc 🌈

1.1. Scope 📋✨

okay so here's what this doc is actually covering, bestie 🎀

  • this is a terminology doc for SIP signaling performance benchmarks on black-box measurements of SIP networking devices 🦋 stress conditions and debugging scenarios are NOT part of the tea here 🙅‍♀️
  • the DUT has to be network equipment that is RFC 3261 capable 💅 it can be a Registrar, a Redirect Server, or a Stateful Proxy. the intermediary does NOT have to act as a stateless proxy, so no pressure 🫶 a DUT can also be a B2BUA or act like an SBC
  • the Tester plays multiple Emulated Agents (EAs) that start (or answer) SIP messages as session endpoints, and send (or receive) the associated media for established connections 🧚
  • now the media situation 🎶
    • the media performance is NOT being benchmarked 🙈
    • some tests need media, but it's only there so we can watch how the SIP signaling is doing 👀 any test that needs media will note the media characteristics as a condition of the test
    • whether the media streams go through the DUT depends on what type of DUT it is, and both situations are in scope 🌸
    • SIP is often used to set up media streams, but the signaling plane and the media plane are treated as orthogonal (totally separate lanes) in this doc 💜 benchmarks for how well those streams perform are out of scope here and in the companion methodology doc [RFC7502] 😭 tests can run with or without media streams, but whether media is there MUST be noted as a condition of the test, since SIP devices can behave differently depending on it 🔥 even when media IS used, we only benchmark the SIP performance, not the media performance or quality
  • both INVITE and non-INVITE scenarios (registrations) are covered 🍓 benchmarking SIP presence or subscribe-notify extensions is NOT part of this doc 🚫
  • different transports (UDP, TCP, SCTP, or TLS) can be used 🌈 the specific transport MUST be noted as a condition of the test, because performance can vary with it ✨
  • REGISTER and INVITE requests may be challenged or left unchallenged for authentication 🔐 whether they're challenged is a condition of the test and gets recorded with the other parameters that could affect the SIP performance of the device or system under test
  • re-INVITE requests are NOT considered here, because the INVITE benchmarks are based on the dialog the INVITE creates, not on the transactions that happen inside that dialog 💅
  • only session establishment is considered for the performance benchmarks 🫧 session disconnect is out of scope. the reason is that the goal is to find the maximum capacity of the device or system under test, meaning how many simultaneous SIP sessions it can support 👑 BYE requests do get created during the test, and those transactions DO add to the load, so they're already counted in the metric we derive. we are NOT looking for a separate metric for how many BYE transactions a device can handle 🙅‍♀️
  • IMS-specific scenarios aren't considered 📞 but the test cases can be applied with 3GPP-specific SIP signaling and the Proxy-Call Session Control Function (P-CSCF) as the DUT 🧚‍♀️
  • these benchmarks are meant for a laboratory environment and are NOT meant for a production network 🧪 some of them send enough traffic that a denial-of-service attack is possible if you ran them on a production network 💀 so please don't 🙏

2. Terminology 📚💖

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this doc are meant to be read as described in BCP 14, RFC2119 [RFC2119] 🎀 RFC 2119 is what defines how these key words get used, so the intent is as clear as possible for Standards Track docs. this doc uses these keywords, but it is NOT a Standards Track doc, so keep that in mind 🫶

for clarity and continuity, this doc uses the definition template from Section 2 of RFC 1242 [RFC1242] ✨

the term "Device Under Test (DUT)" is defined in Section 3.1.1 of RFC 2285 [RFC2285] 💅

a bunch of the SIP terms in this doc are defined in RFC 3261 [RFC3261], so here are the most important ones for convenience 🌸 using these terms here is consistent with their definitions in the base SIP spec [RFC3261], as amended by [RFC4320], [RFC5393], and [RFC6026] 💜

  • Call Stateful: a proxy is call stateful if it keeps state for a dialog from the initiating INVITE all the way to the terminating BYE request 🦋 a call stateful proxy is always transaction stateful, but it doesn't work the other way around 👀
  • Stateful Proxy: a logical entity (as defined by [RFC3261]) that keeps the client and server transaction state machines going while it processes a request 🧚 (also known as a transaction stateful proxy.) the behavior of a stateful proxy is further defined in Section 16 of RFC 3261 [RFC3261] . a transaction stateful proxy is NOT the same thing as a call stateful proxy 🙃
  • Back-to-Back User Agent: a back-to-back user agent (B2BUA) is a logical entity that receives a request and handles it as a user agent server (UAS) 🎀 to figure out how the request should be answered, it acts as a user agent client (UAC) and generates requests 🔄 unlike a proxy server, it keeps dialog state and MUST participate in all requests sent on the dialogs it has established 🔥 since it's just a UAC and a UAS glued together, no explicit definitions are needed for its behavior 💅

3. Term Definitions 🧸

3.1. Protocol Components 🔌

3.1.1. Session 💬

Definition: the combo of signaling and media messages, plus the processing that goes with them, that enables a single SIP-based audio or video call, or a SIP registration ✨

Discussion: "session" usually implies a media session 🎶 in this doc the term gets extended to cover the signaling AND any media that the signaling specifies and invokes 🫶

Measurement Units: N/A.

Issues: None.

See Also:

  • Media Plane
  • Signaling Plane
  • Associated Media

3.1.2. Signaling Plane 📨

Definition: the plane where SIP messages [RFC3261] get exchanged between SIP agents [RFC3261] 💌

Discussion: SIP messages set up sessions a few different ways 🛤️ directly between two User Agents [RFC3261], through a Proxy Server [RFC3261], or through a chain of Proxy Servers 🌈 the Session Description Protocol (SDP) lives in the Signaling Plane 📝

Measurement Units: N/A.

Issues: None.

See Also:

  • Media Plane
  • Emulated Agent

3.1.3. Media Plane 🎧

Definition: the data plane where one or more media streams, and their associated media control protocols (e.g., RTCP [RFC3550]), get exchanged between User Agents after a media connection has been created by exchanging signaling messages in the Signaling Plane 🎬

Discussion: media can also be called the "bearer channel" 🎀 the Media Plane MUST include the media control protocol, if one is used, plus the media stream(s) 🔥 examples of media are audio and video 🎶 the media streams are described in the SDP of the Signaling Plane 📝

Measurement Units: N/A.

Issues: None.

See Also:

  • Signaling Plane

3.1.4. Associated Media 🔗

Definition: media that corresponds to an 'm' line in the SDP payload of the Signaling Plane 📄

Discussion: the format of the media is set by the SDP attributes for that corresponding 'm' line 🧩

Measurement Units: N/A.

Issues: None.

3.1.5. Overload 🥵

Definition: overload is the state where a SIP server doesn't have enough resources to process all incoming SIP messages [RFC6357] 😭

Discussion: telling an overload apart from other failure scenarios is outside the scope of black-box testing and of this doc 🙈 under overload, all or a percentage of Session Attempts will fail because there aren't enough resources 💀 in black-box testing, the cause of the failure isn't explored 🕵️‍♀️ the fact that a failure happened, for whatever reason, is what makes the tester lower the offered load, as described in the companion methodology doc [RFC7502] 📉 SIP server resources can include CPU processing capacity, network bandwidth, input/output queues, or disk resources 💻 any combination of these can be fully used up when a SIP server (the DUT) is overloaded 🔥 for proxy-only (or intermediary) devices, the proxy is expected to be driven into overload based on the delivery rate of signaling requests 🚀

Measurement Units: N/A.

3.1.6. Session Attempt 📤

Definition: a SIP INVITE or REGISTER request sent by the EA that has NOT received a final response 🎯

Discussion: the attempted session can be either an invitation to an audio/video communication or a registration attempt 📞 when counting session attempts, we include ALL requests that get rejected for lack of authentication information 🔐 the EA needs to record the total number of session attempts, INCLUDING the ones a proxy routinely rejects because it requires the UA to authenticate itself 🙃 the EA is provisioned to deliver a specific number of session attempts per second 🏃‍♀️ but the EA MUST also count the actual number of session attempts per given time interval 🧮

Measurement Units: N/A.

Issues: None.

See Also:

  • Session
  • Session Attempt Rate

3.1.7. Established Session ✅

Definition: a SIP session for which the EA acting as the UA has received a 200 OK message 🥂

Discussion: an Established Session can be either an invitation to an audio/video communication or a registration attempt 🎉 early dialogs for INVITE requests are out of scope for this work 🙅‍♀️

Measurement Units: N/A.

Issues: None.

See Also: None.

3.1.8. Session Attempt Failure ❌

Definition: a session attempt that does NOT result in an Established Session 💔

Discussion: the session attempt failure can be indicated by the following observations at the EA 👀

  1. receipt of a SIP 3xx-, 4xx-, 5xx-, or 6xx-class response to a Session Attempt 📭
  2. the lack of ANY received SIP response to a Session Attempt within the Establishment Threshold Time (cf. Section 3.3.2) ⏳

Measurement Units: N/A.

Issues: None.

See Also:

  • Session Attempt

3.2. Test Components 🧪

3.2.1. Emulated Agent ✨

Definition: okay so an EA is a device in the test topology that starts or answers SIP messages as one or more session endpoints 📞 and, wherever it makes sense, sends or receives Associated Media for Established Sessions 🎧

Discussion: the EA lives in both the Signaling and Media Planes 💅 and the Tester is allowed to act as multiple EAs 🧚

Measurement Units: N/A

Issues: None

See Also:

  • Media Plane
  • Signaling Plane
  • Established Session
  • Associated Media

3.2.2. Signaling Server 🎀

Definition: this is the device in the test topology that helps set up sessions between EAs 🫶 and it is the DUT, bestie

Discussion: the DUT is a network intermediary that's RFC 3261 capable 💖 so think Registrar, Redirect Server, Stateful Proxy, B2BUA, or SBC. main character energy 👑

Measurement Units: N/A

Issues: None

See Also:

  • Signaling Plane

3.2.3. SIP Transport Protocol 🦋

Definition: the protocol used to carry the Signaling Plane messages 🚚

Discussion: ngl this one is so important 😭 the same SIP networking device can have different performance benchmarks depending on whether it's running over TCP, UDP, TLS, SCTP, websockets [RFC7118], or any future transport-layer protocol 🌈 so we have to measure the SIP Performance Benchmarks with all these transports. Performance Benchmarks MUST report the SIP Transport Protocol used to get the benchmark results 📝

Measurement Units: not actual units of measure, but they're attributes that are one of many factors shaping the measurements you'll take 🍓 TCP, UDP, SCTP, TLS over TCP, TLS over UDP, TLS over SCTP, and websockets are all possible values to record during the test 💜

Issues: None

See Also: None

3.3. Test Setup Parameters 🌸

3.3.1. Session Attempt Rate ⭐

Definition: this is the EA config for how many sessions per second (sps) the EA tries to set up using the DUT's services 🚀

Discussion: the Session Attempt Rate is the number of sessions per second the EA sends toward the DUT. Not every attempted session is going to end up established, and that's just life 🥲

Measurement Units: Session Attempts per second

Issues: None

See Also:

  • Session
  • Session Attempt

3.3.2. Establishment Threshold Time ⏰

Definition: this is the EA config for how long an EA client will wait for a response from an EA server before it calls it a Session Attempt Failure 🙈

Discussion: this time duration depends on the test 💅

It is RECOMMENDED that the Establishment Threshold Time value be set to Timer B or Timer F as specified in RFC 3261, Table 4 [RFC3261] 🫶

Measurement Units: seconds

Issues: None

See Also: None

3.3.3. Session Duration 🕰️

Definition: this is the EA config for how long the SIP dialog is supposed to last between the two EAs in the test ✨

Discussion: when the BYE gets sent is what controls the Session Duration. So timing is everything 💀

Measurement Units: seconds

Issues: None

See Also: None

3.3.4. Media Packet Size 🎶

Definition: this is the config on the EA for a fixed number of frames or samples sent in each RTP packet of the media stream, for when the test has Associated Media 🎧

Discussion: this document is about measuring SIP performance 💖 but if the DUT is processing media as well as SIP messages, the media processing can slow down the SIP processing and lower the SIP performance metric 🐢 The tests with associated media are designed for audio codecs, and the assumption was that bigger media packets need more processor time 🧠 This document does not define parameters for video codecs, so we're not going there 🙅‍♀️

For a single benchmark test, media sessions use a defined number of samples or frames per RTP packet. If two SBCs, for example, used the same codec but one put more frames into the RTP packet, that could cause variation in the performance benchmark results 👀

Measurement Units: an integer number of frames or samples, depending on whether a hybrid- or sample-based codec is used, respectively 🍰

Issues: None

See Also: None

3.3.5. Codec Type 🎤

Definition: the name of the codec used to generate the media session 💜

Discussion: for a single benchmark test, every session uses the same size packet for media streams 📦 and the size of the packets can cause variation in the performance benchmark measurements 🙌

Measurement Units: this is a textual name (alphanumeric) that uniquely identifies the codec 🔤

Issues: None

See Also: None

3.4. Benchmarks 🏆

3.4.1. Session Establishment Rate 🚦

Definition: the maximum Session Attempt Rate the DUT can handle for an extended, predefined period with zero failures 🔥

Discussion: this benchmark is obtained with zero failures 🙅‍♀️ The Session Attempt Rate on the EA gets raised and lowered following the algorithm in the companion methodology document [RFC7502] until a traffic load finishes the time it takes to attempt N sessions without any failures 😤 N is a parameter in the algorithm and gets recorded in the Test Setup Report 📋

Measurement Units: sessions per second (sps)

Issues: None

See Also:

  • Session Attempt Rate

3.4.2. Registration Rate 📇

Definition: the maximum Registration Attempt Rate the DUT can handle for an extended, predefined period with zero failures ✨

Discussion: this benchmark is obtained with zero failures 💯 The registration rate provisioned on the Emulated Agent gets raised and lowered following the algorithm in the companion methodology document [RFC7502] until a traffic load of registration attempts at the given attempt rate finishes the time it takes to attempt N registrations without failure 🫠 N is a parameter in the algorithm and gets recorded in the Test Setup Report 📋

This benchmark is described separately from the Session Establishment Rate (Section 3.4.1) even though it could count as a special case of that one 🤓 A REGISTER request is a request for a session that isn't started by an INVITE request, but it gets its own benchmark because it's super important for most SIP installations 💅 For example, think about an avalanche restart, where hundreds of thousands of endpoints register at the same time after a power outage ⚡ In that case, an accurate measurement of how well the device can register endpoints is really useful for the network designer 🧑‍💻 Also, in certain controlled networks there seems to be a difference between the registration rate of new endpoints and the registering rate of existing endpoints (register refreshes) 👀 This benchmark can catch those differences too 🔍

Measurement Units: registrations per second (rps)

Issues: None

See Also: None

3.4.3. Registration Attempt Rate 📤

Definition: the EA config for how many registrations per second the EA tries to send to the DUT 🎯

Discussion: the Registration Attempt Rate is the number of registration requests per second the EA sends toward the DUT 💌

Measurement Units: registrations per second (rps)

Issues: None

See Also: None

4. Security Considerations 🔐

Documents of this type don't directly affect the security of the Internet or corporate networks, as long as benchmarking isn't performed on devices or systems connected to production networks 🚨 Security threats and how to counter them in SIP and the media layer are discussed in RFC 3261 [RFC3261], RFC 3550 [RFC3550], and RFC 3711 [RFC3711]. This document attempts to formalize a set of common terminology for benchmarking SIP networks. Packets with unintended and/or unauthorized DSCP or IP precedence values may present security issues 💀 Figuring out the security consequences of those packets is out of scope for this document, so that's a whole different story 🙈

5. References 📚

5.1. Normative References

5.2. Informative References

Acknowledgments 🙏✨

Okay bestie, the people who made this doc happen deserve all the flowers 💐. Huge thanks to Keith Drage, Cullen Jennings, Daryl Malas, Al Morton, and Henning Schulzrinne for their invaluable contributions 🫶. Dale Worley did an extensive review that led to improvements in the documents, which is so iconic 👑. The authors are also grateful to Barry Constantine, William Cerveny, and Robert Sparks for their valuable comments during the documents' last calls and expert reviews 💜. And Al Morton and Sarah Banks were exemplary working group chairs 🌟. We thank them for tracking this work to completion, periodt 💅.

Authors' Addresses 🎀

Here's where you can find the authors 💌. Carol Davids is at Illinois Institute of Technology 🎓:

Carol Davids
Illinois Institute of Technology
201 East Loop Road
Wheaton, IL  60187
United States

Phone: +1 630 682 6024
EMail: davids@iit.edu

Vijay K. Gurbani is at Bell Laboratories, Alcatel-Lucent 🔬:

Vijay K. Gurbani
Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane
Rm 9C-533
Naperville, IL  60566
United States

Phone: +1 630 224 0216
EMail: vkg@bell-labs.com

Scott Poretsky is at Allot Communications 🦋:

Scott Poretsky
Allot Communications
300 TradeCenter, Suite 4680
Woburn, MA  08101
United States

Phone: +1 508 309 2179
EMail: sporetsky@allot.com
📦 where this page comes from