RFC 6076
Basic Telephony SIP End-to-End Performance Metrics
RFC 6076: Basic Telephony SIP End-to-End Performance Metrics 🎀
okay bestie, so this is RFC 6076 and it's giving standards-track energy ✨. It's titled Basic Telephony SIP End-to-End Performance Metrics and it came out in January 2011. The authors are D. Malas from CableLabs and A. Morton from AT&T Labs. It's marked as Standards Track, and the ISSN is 2070-1721. The title block doesn't list any documents it obsoletes or updates, so it's not replacing or changing anything else 🧚
Abstract 💜
okay so the whole point of this doc is giving you a defined set of metrics for measuring how well end-to-end SIP (Session Initiation Protocol) performs for telephony services. That's true whether it's running in production or in testing. Everyone using the same standard set of metrics means the results are interoperable, which makes it way easier to compare what different industry implementations are doing 🫶
Status of This Memo 🌸
this is an Internet Standards Track document. It's a product of the IETF (Internet Engineering Task Force), which means it reflects the consensus of the IETF community. It went through public review and got approved for publication by the IESG (Internet Engineering Steering Group). If you want more info on Internet Standards, check Section 2 of RFC 5741.
For the current status of this document, any errata, and how to send feedback, head to http://www.rfc-editor.org/info/rfc6076 🔗
Copyright Notice 🌈
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.
so this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at http://trustee.ietf.org/license-info as they stood on the publication date. Please actually read those, bestie, because they spell out your rights and restrictions for this doc 📜. Any Code Components pulled out of this document have to include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They also come with no warranty, which is also described in the Simplified BSD License.
this document may include material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. Some of the people who hold the copyright on that older material might not have given the IETF Trust permission to allow changes outside the IETF Standards Process. So unless you get a proper license from those copyright holders, this document can't be modified outside the IETF Standards Process, and you can't make derivative works of it outside that process either. The only exceptions are formatting it for publication as an RFC or translating it into languages other than English 💅
Table of Contents 📑
1. Introduction and Scope ..........................................3
2. Terminology .....................................................4
3. Time Interval Measurement and Reporting .........................5
4. SIP Performance Metrics .........................................7
4.1. Registration Request Delay (RRD) ...........................8
4.2. Ineffective Registration Attempts (IRAs) ...................9
4.3. Session Request Delay (SRD) ...............................10
4.3.1. Successful Session Setup SRD .......................11
4.3.2. Failed Session Setup SRD ...........................12
4.4. Session Disconnect Delay (SDD) ............................13
4.5. Session Duration Time (SDT) ...............................15
4.5.1. Successful Session Duration SDT ....................15
4.5.2. Failed Session Completion SDT ......................17
4.6. Session Establishment Ratio (SER) .........................18
4.7. Session Establishment Effectiveness Ratio (SEER) ..........19
4.8. Ineffective Session Attempts (ISAs) .......................20
4.9. Session Completion Ratio (SCR) ............................21
5. Additional Considerations ......................................23
5.1. Metric Correlations .......................................23
5.2. Back-to-Back User Agent (B2BUA) ...........................23
5.3. Authorization and Authentication ..........................23
5.4. Forking ...................................................24
5.5. Data Collection ...........................................24
5.6. Testing Documentation .....................................25
6. Conclusions ....................................................25
7. Security Considerations ........................................25
8. Contributors ...................................................26
9. Acknowledgements ...............................................26
10. References ....................................................26
10.1. Normative References .....................................26
10.2. Informative References ...................................271. Introduction and Scope ✨
okay so SIP has become super popular with service providers, vendors, and end users across the telecom industry 💖. There are plenty of standards for measuring the performance of telephony signaling protocols, like Signaling System 7 (SS7), but none of them specifically cover SIP 🙈
this doc's scope is locked into defining a standard set of metrics for measuring and reporting SIP performance from an end-to-end perspective in a telephony environment. The metrics give everyone a shared foundation for understanding and quantifying performance expectations between service providers, vendors, and the people using SIP-based services. The main audience is network operators, who are always collecting info on how responsive the network is to customer requests for services 🦋
measurements of these metrics get affected by stuff outside of SIP. Here's a non-exhaustive list of examples:
- Network connectivity
- Switch and router performance
- Server processes and hardware performance
this doc defines a list of useful metrics for different parts of a telephony environment. You can use them one at a time or as a set, depending on how SIP is being used in a given telecom service 🎀
the metrics in this document DO NOT take into account impairment or failure of the actual application processing of a request or response. They also can't tell application processing time apart from other sources of delay, like packet transfer delay 😭
metrics for quantifying the application processing performance of a single device are out of scope for this document 🙅♀️
this document doesn't give any numerical objectives or acceptance threshold values for the SIP performance metrics defined below, since those are generally beyond what IETF activities cover 💅
the metrics here apply in two kinds of scenarios: when the SIP messages sent into a network under test are dedicated messages just for testing, or when the messages are user-initiated and some of the live traffic is present. People sometimes call these active and passive measurement, respectively 🧚
2. Terminology 💅
here's the lowdown on the terms and conventions we'll be using all throughout this doc 💜
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].
End-to-End means two or more elements that get used to start a request, receive that request, and respond to it. It covers all the elements needed to be part of a session dialog between the originating user agent client (UAC), the destination user agent server (UAS), and any interim proxies (which may also include back-to-back user agents (B2BUAs)). This could be limited to one operator's set of elements, or it could stretch across every element involved with a session, if that goes beyond a single operator's network 🌸
Session: as RFC 3261 [RFC3261] describes it, SIP is mainly used to request, create, and conclude sessions. "These sessions include Internet telephone calls, multimedia distribution, and multimedia conferences". The metrics in this doc measure the performance of the SIP dialogs needed to set up these sessions, which is why they have names like Session Request Delay and Session Disconnect Delay 🎀. Even though a lot of the metric names include the word "session", they only measure the signaling side of things. Each session is identified by a unique "Call-ID", "To", and "From" header field tag ✨
Session Establishment happens when a 200 OK response has been received from the target UA, replying to the originating UA's INVITE setup request. That tells you the session setup request worked 🔥
Session Setup: as referenced in the sub-sections of Section 4.2 in this document, session setup is the set of messages and included parameters directly related to a UA asking to establish a session with a corresponding UA. It's also described as a series of steps needed to establish "ringing" [RFC3261] 🔔
3. Time Interval Measurement and Reporting ✨
okay so a bunch of the metrics in this memo use a clock to figure out how much time passed between two events 🕰️💖 this section is where we pin down the time-related terms and what has to get reported 🎀
t1 - start time 🚀
This is the moment a request gets sent, and it's where the continuous time interval begins. t1 happens once the designated request has been processed by the SIP application and the first bit of the request packet has left the UA or proxy (and you can actually observe it at some logical or physical interface) 💅
t1 marks the start of each request-response test, and it SHALL be used to designate the time of day when a particular measurement was conducted. For example, the Session Request Delay at "t1" (at some specific UA interface) was measured to be X ms 🌸
t4 - end time 🏁
This is the moment that wraps up the continuous time interval that started with the related request. t4 happens when the last bit of the designated response gets received by the SIP application at the requesting device (and again, it's observable at some logical or physical interface) 🫶
Note: The designations t2 and t3 are reserved for future use at another interface involved in satisfying a request.
Section 10.1 of [RFC2330] goes over the time-related issues in measurements and defines the errors that can come from the clocks themselves. These definitions get used in the stuff below 🧚
Time-of-Day Accuracy
okay this is the part where we talk about how close to real time our clock is 🕐✨ t1 is tied to the start of a request, and it also doubles as the time-of-day stamp for one specific measurement. The clock offset [RFC2330] is the gap between t1 and a recognized primary source of time, like UTC (offset = t1 - UTC) 💜
when measurement results are going to be correlated with other results or information using time-of-day stamps, the clock that supplies t1 SHOULD be synchronized to a primary time source, so the clock's offset stays small 🎯 the clocks at the different measurement points SHOULD also be synchronized with each other, so the relative offset (as defined in RFC2330) stays small too 🤝 and the clock's offset and the relative offset MUST be reported with each measurement. no skipping this one bestie 🙅♀️💅
Time Interval Accuracy
the accuracy of the t4-t1 interval is just as important to keep up and report 📏 the difference between a clock's offsets at t1 and t4 is one source of error for the measurement, and it's tied to the clock's skew [RFC2330] 🌈
a stable and reasonably accurate clock is needed to make the time interval measurements this memo asks for ⏱️ this source of error SHOULD be constrained to less than +/- 1 ms, which works out to 1-part-per-1000 frequency accuracy for a 1-second interval. the longer the t4-t1 gets, the more stability you need to keep that error under +/- 1 ms, so yeah, longer intervals are a whole commitment 😭
there are some other important things about how clocks operate too 👑
- Synchronization protocols need to be able to adjust the local clock. The catch is that these adjustments (clock steps or slewing) can cause big errors if they happen during the t1 to t4 measurement interval 🙈 Clock correction SHOULD be suspended during a t1 to t4 measurement interval, unless the time interval accuracy requirement above will be met. Otherwise, a measurement SHOULD NOT be performed during clock correction, unless the time interval accuracy requirement above will be met.
- If a free-running clock is used to make the time interval measurement, then the time of day reported with the measurement (which is normally timestamp t1) SHOULD be derived from a different clock that meets the time-of-day accuracy requirements described above 💖
reading time off a clock can be limited by the delay it takes to service the interrupt 🥹 so if the accuracy of the time stamp read at t1 or t4 includes the interrupt delay, this source of error SHOULD be known and included in the error assessment 📝
4. SIP Performance Metrics 🎀
okay this is the big one where we get into the actual metrics 🦋 for every metric in this section, t1 starts with the first associated SIP message sent by either UA. it does NOT reset if the UA has to retransmit the same message multiple times within the same transaction. that first message is what matches up with what the user or application expects from the request 💅
some metrics look across different transactions so they can measure stuff like redirection and failure recovery 🔁 the end time usually comes from a successful end-to-end provisional response, a successful final response, or a failure final response where there's no recovery. each individual metric tells you which message to use for the end time 🧚
the authentication method used to set up the SIP dialog changes the message exchanges 🔐 the examples here don't try to cover every authentication type. SIP Digest authentication got picked for the examples since authentication shows up a lot 💜
for all of the metrics, how accurate and granular the output values are depends on how accurate and granular the input values are 📊 a few of the metrics below are defined as a ratio, and when the denominator of that ratio is 0, the metric is undefined 🙅♀️
these metrics don't specify a sample size, but you should still keep it in mind 👀 bigger sample sets give you a better read on performance. for example, some SIP Service Providers (SSPs) [RFC5486] might collect input over an hourly, daily, weekly, or monthly timeframe, while another SSP might run the calculations over a changing set of SIP dialogs 🌸
4.1. Registration Request Delay (RRD) 🌷
Registration Request Delay (RRD) measures how long it takes to respond to a UA REGISTER request ⏳ RRD SHALL be measured and reported only for successful REGISTER requests, and Ineffective Registration Attempts (Section 4.2) SHALL be reported for failures 💅 this metric is measured at the originating UA. the output value is numerical and SHOULD be stated in units of milliseconds 🔢 RRD is calculated with this formula:
RRD = Time of Final Response - Time of REGISTER Requestin a successful registration attempt, RRD is the time from when the first bit of the initial REGISTER message (the one with the necessary info) gets passed by the originating UA to the intended registrar, until the last bit of the 200 OK comes back and tells you the registration finished successfully 🥳 this dialog includes an expected authentication challenge before the 200 OK shows up, like in the registration flow examples below 🔐
here's an example of the identifiable events you need as inputs for calculating RRD during a successful registration 🦋
UA1 Registrar
| |
|REGISTER |
t1---->|--------------------->|
/\ | 401|
|| |<---------------------|
RRD |REGISTER |
|| |--------------------->|
\/ | 200|
t4---->|<---------------------|
| |Note: Networks with elements using primarily Digest authentication will exhibit different RRD characteristics than networks with elements primarily using other authentication mechanisms (such as Identity). Operators monitoring RRD in networks with a mixture of authentication schemes should take note that the RRD measurements will likely have a multimodal distribution. 🙈
4.2. Ineffective Registration Attempts (IRAs) 🌈
Ineffective registration attempts are used to catch failures or impairments that stop a registrar from receiving a UA REGISTER request 🚨 this metric is measured at the originating UA. the output value is numerical and SHOULD be reported as a percentage of registration attempts 📉
this metric is calculated as a percentage of total REGISTER requests. the IRA percentage uses this formula:
# of IRAs
IRA % = ----------------------------- x 100
Total # of REGISTER Requestsa failed registration attempt is a final failure response to the initial REGISTER request 💀 it usually means a failure came back from the destination registrar or an interim proxy, or the REGISTER request timed out at the originating UA. a failure response is a 4XX (excluding 401, 402, and 407 non-failure challenge response codes), a 5XX, or possibly a 6XX message. a timeout failure is spotted when Timer F expires ⏰ IRAs can help detect problems in downstream signaling functions that might be keeping the REGISTER message from reaching the intended registrar, or they can show that a registrar is overloaded and can't respond to the request 🫶
here's a timeout example of the identifiable event you need as input for a failed registration attempt 🧚
UA1 Registrar
| |
|REGISTER |
|--------------------->|
|REGISTER |
|--------------------->|
|REGISTER |
|--------------------->|
| |
Failure ---->|***Timer F Expires |
| |in that exchange, UA1 retries a REGISTER request multiple times before the timer expires, which is what tells you it failed 🔁 only the first REGISTER request MUST be used as input to the calculation and for an IRA. the later REGISTER retries carry the same transaction identifier (the same topmost Via header field branch parameter value), and they MUST be ignored for metric calculation purposes. this is what keeps the metric output accurate 🎯✨
and here's a registrar servicing failure example of the identifiable event you need as input for a failed registration attempt 💜
UA1 Registrar
| |
|REGISTER |
|--------------------->|
| |
| |
| |
| |
| 503|
Failure ---->|<---------------------|
| |4.3. Session Request Delay (SRD) ✨
Okay bestie, SRD is how we catch the stuff that makes a session take forever to get a response 🐢💤. We measure it for both the sessions that succeed and the ones that fail, because it's basically the user's experience in a nutshell 🫶. Important: failed-request SRD MUST NOT get mixed into the same result as successful requests 🙅♀️. Those two durations are gonna be wildly different, and the target time for each is usually way different too, so keeping them separate is just the vibe 💅. This metric is kinda like Post-Selection Delay from [E.721] 🎀, and it's only measured at the originating UA 📍. The output needs to say whether it's for successful or failed session requests 🏷️, and it SHOULD be in seconds ⏱️. The math is:
SRD = Time of Status Indicative Response - Time of INVITE4.3.1. Successful Session Setup SRD 🦋
For a successful attempt, SRD runs from the moment the first bit of the initial INVITE (the one with all the necessary info) leaves the originating user agent heading to the mediation or destination agent, until the last bit of the first provisional response arrives. That response shows an audible or visual status for the setup 🔔. (Note: the initial INVITE might get forked 🍴. Section 5.4 has some things to think about on forking, so go check it out 👀.) In SIP, the status message is a non-100 Trying provisional message that comes back in reply to an INVITE. Sometimes there's no non-100 Trying provisional, and a 200 shows up first instead 🙃. In those cases, the 200 is what gets used for the calculation. Mostly though, this metric relies on getting a non-100 Trying message 💌. Using the Provisional Response ACKnowledgement (PRACK) method [RFC3262] MAY make the results better and more consistent 📈.
Here's a message exchange showing the events you need for SRD in a successful setup without a redirect (aka no 3XX message) 📨:
UA1 UA2
| |
|INVITE |
t1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 180|
t4---->|<---------------------|
| |And here's one for a successful setup with a redirect (like a 302 Moved Temporarily) 🔀:
UA1 Redirect Server UA2
| | |
|INVITE | |
t1---->|--------------------->| |
/\ | 302| |
|| |<---------------------| |
|| |ACK | |
SRD |--------------------->| |
|| |INVITE |
|| |------------------------------------------->|
\/ | 180|
t4---->|<-------------------------------------------|4.3.2. Failed Session Setup SRD 💔
For a failed attempt, SRD is the time from when the first bit of the initial INVITE leaves the originating agent or user, heading to the intended mediation or destination agent, until the last bit of either the first provisional response or a failure indication response 😤. A failure response is a 4XX (but NOT the 401, 402, and 407 non-failure challenge response codes 🙅♀️), a 5XX, or possibly a 6XX message. If this metric changes, that could mean there's trouble in downstream signaling functions 🚧. Maybe the INVITE isn't getting to the intended UA, or the endpoint behavior just changed 🤔. This metric measures the delay for a failed session request, but if you want a ratio of session attempt failures, that's what Ineffective Session Attempts (Section 4.8) is for 📊.
Here's the failed setup without a redirect (no 3XX) 🥀:
UA1 UA2
| |
|INVITE |
t1---->|--------------------->|
/\ | |
|| | |
SRD | |
|| | |
\/ | 480|
t4---->|<---------------------|
| |And the failed setup with a redirect (like 302 Moved Temporarily) 🌀:
UA1 Redirect Server UA2
| | |
|INVITE | |
t1---->|--------------------->| |
/\ | 302| |
|| |<---------------------| |
|| |ACK | |
SRD |--------------------->| |
|| |INVITE |
|| |------------------------------------------->|
\/ | 480|
t4---->|<-------------------------------------------|4.4. Session Disconnect Delay (SDD) 📴
This one catches failures or slowdowns that take too long to end a session 🛑. SDD is measured for successful and failed session disconnects, but SDD for disconnects that end in a failure MUST NOT be combined with successful ones in the same result 🙅♀️. Same energy as before: the durations and the target times are gonna be super different 🙈. You can measure it from either end-point UA in the SIP dialog 🔁. The output is a number, and it SHOULD be in milliseconds ⚡. The formula is:
SDD = Time of 2XX or Timeout - Time of Completion Message (BYE)So SDD is the gap between the first bit of the completion message you sent (like a BYE 👋) and the last bit of the 2XX response that comes back ✅. Sometimes you get a recoverable error response first, like a 503 Retry-After 🔄. In that case, those responses should not be used as the end time. Instead, you use the successful (2XX) response to the recovery message 🙌. These message exchanges show the events you need for SDD during a successful session completion 📝:
Measuring SDD at the originating UA (UA1) -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
|BYE |
t1---->|--------------------->|
/\ | |
|| | |
SDD | |
|| | |
\/ | 200|
t4---->|<---------------------|Measuring SDD at the target UA (UA2) -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|
|ACK |
|--------------------->|
| BYE|
|<---------------------|<----t1
| | /\
| | ||
| | SDD
| | ||
|200 | \/
|--------------------->|<----t4Sometimes, no response comes back after the completion message is sent (and maybe retried) 👻. Then the BYE ends up hitting a Timer F expiration ⏰. Sessions that end like this SHOULD be excluded from the calculation 🚫.
4.5. Session Duration Time (SDT) ⏳
SDT helps us spot problems (like bad audio quality 🎧📉) that make sessions way too short 💨. It's measured for both successful and failed session completions 🫶. You can measure it from either end-point UA in the SIP dialog 🔁. This one's kinda like Call Hold Time, and in telephony apps of SIP it's usually calculated as Average Call Hold Time (ACHT) 📞. The output is a number, and it SHOULD be in seconds ⏱️. The formula is:
SDT = Time of BYE or Timeout - Time of 200 OK response to INVITEReal talk: this metric does NOT measure sessions that use early media 🙅♀️. For example, some automated response systems only use early media. They send a SIP 183 Session Progress message with Session Description Protocol (SDP) that connects the originating UA to the automated message 🤖. In those sessions, the originating UA usually never gets a 200 OK, and the exchange ends with the originating UA sending a CANCEL 🥲.
4.5.1. Successful Session Duration SDT 🎀
Okay bestie, so for a successful session, SDT gets averaged ✨. It's the length of a dialog, measured from the moment the first bit of a 200 OK response to an INVITE arrives until the last bit of the matching BYE arrives, the one that says the dialog is complete 💅. Retransmissions of the 200 OK and ACK caused by network impairments do NOT reset the metric timers, so the clock keeps ticking no matter how messy the network gets 🫶.
Here's how the identifiable events line up. The message exchanges swap between the originating and target UAs just to show different angles 🦋.
Measuring SDT at the originating UA (UA1) -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
t1---->|<---------------------|
/\ |ACK |
|| |--------------------->|
|| | |
SDT | |
|| | |
|| | |
\/ | BYE|
t4---->|<---------------------|
| |When measuring SDT at the target UA (UA2), it's the interval between sending the first bit of a 200 OK response to an INVITE and receiving the last bit of an associated BYE message that says the dialog is done 🌸. If UA2 is the one that starts the BYE, then it's the interval between sending the first bit of the 200 OK to an INVITE and sending the first bit of that associated BYE message 🧚. Here's an example:
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|<----t1
|ACK | /\
|--------------------->| ||
| | ||
| | SDT
| | ||
| | ||
| BYE| \/
|<---------------------|<----t4
| |(In these two examples, t1 is the same even if either UA receives the BYE instead of sending it.) 🙈
4.5.2. Failed Session Completion SDT 💀
Sometimes no response comes back after a session completion message is sent (and possibly retried) 😭. In that case, SDT is defined as the interval between receiving the first bit of a 200 OK response to an INVITE and the resulting Timer F expiration ⭐.
Measuring SDT at the originating UA (UA1) -
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
t1---->|<---------------------|
/\ |ACK |
|| |--------------------->|
|| |BYE |
SDT |--------------------->|
|| |BYE |
|| |--------------------->|
\/ | |
t4---->|***Timer F Expires |When measuring SDT at UA2, it's defined as the interval between sending the first bit of a 200 OK response to an INVITE and the resulting Timer F expiration. Here's the example flow 🦋:
UA1 UA2
| |
|INVITE |
|--------------------->|
| 180|
|<---------------------|
| 200|
|<---------------------|<----t1
| ACK| /\
|--------------------->| ||
| BYE| ||
|<---------------------| SDT
| BYE| ||
|<---------------------| ||
| | \/
| Timer F Expires***|<----t4Heads up: with message loss and retransmission in the mix, the value measured at UA1 can differ from the value measured at UA2, by up to the value of Timer F 👀.
4.6. Session Establishment Ratio (SER) 💖
This metric is all about checking whether a terminating UA or downstream proxy can actually establish sessions for new session INVITE requests 🎯. SER is the ratio of new session INVITE requests that get a 200 OK response to the total number of attempted INVITE requests, minus the INVITE requests that got a 3XX response 💅. It's similar to the Answer Seizure Ratio (ASR) defined in [E.411] 🌈. It's measured at the originating UA only. The output is a number that SHOULD be adjusted so it reads as a percentage of successfully established sessions ✨. The SER is calculated with this formula:
# of INVITE Requests w/ associated 200 OK
SER = --------------------------------------------------------- x 100
(Total # of INVITE Requests) -
(# of INVITE Requests w/ 3XX Response)Here's an example of the identifiable events you need to figure out session establishment the way described above 🫶:
UA1 UA2
| |
|INVITE |
+----------->|------------------>|
| | 180|
| |<------------------|
Session Established | |
| | |
| | 200|
+----------->|<------------------|
| |And here's a message exchange that includes a SIP 302 Redirect response 🧚:
UA1 UA2 UA3
| | |
|INVITE | |
+----------->|------------------>| |
| | | |
INVITE w/ 3XX Response | | |
| | 302| |
+----------->|<------------------| |
| | |
|INVITE |
+----------->|-------------------------------------->|
| | |
| | 180|
Session Established |<--------------------------------------|
| | |
| | 200|
+----------->|<--------------------------------------|
| |4.7. Session Establishment Effectiveness Ratio (SEER) 🌸
This metric is complementary to SER, but it's designed to leave out the potential effects of an individual user of the target UA 🙈. SEER is the ratio of INVITE requests that get a 200 OK response, plus INVITE requests that get a 480, 486, 600, or 603, to the total number of attempted INVITE requests, minus the INVITE requests that got a 3XX response ✨. The response codes 480, 486, 600, and 603 were picked because they clearly point to the effect of an individual user of the UA 💜.
It's possible that an individual user could cause a negative effect on the UA. For example, they might have misconfigured the UA, which causes a response code that isn't directly related to an SSP. But that can't be easily determined from an intermediary B2BUA sitting somewhere between the originating and terminating UAs 🤔. With that in mind, response codes like 401, 407, and 420 (this isn't an exhaustive list) were NOT included in the numerator of the metric 💅.
This metric is similar to the Network Effectiveness Ratio (NER) defined in [E.411] 🌈. It's measured at the originating UA only. The output is a number that SHOULD be adjusted so it reads as a percentage of successfully established sessions, minus common UAS failures 👑.
The SEER is calculated with this formula:
SEER =
# of INVITE Requests w/ associated 200, 480, 486, 600, or 603
------------------------------------------------------------- x 100
(Total # of INVITE Requests) -
(# of INVITE Requests w/ 3XX Response)Reference the example flows in Section 4.6 🦋.
4.8. Ineffective Session Attempts (ISAs) ✨
Okay so ISAs happen when a proxy or agent internally lets go of a setup request because it's failed or overloaded 😭. This one's kinda like Ineffective Machine Attempts (IMAs) in SIP telephony apps, and it was borrowed from Telcordia GR-512-CORE [GR-512] 💅. The output is numerical and it SHOULD be adjusted to show a percentage 🌸. These failure responses are the guideline:
408 Request Timeout500 Server Internal Error503 Service Unavailable504 Server Time-out
This set was put together the same way as in Section 4.7 🎀. Also, 408 can mean a downstream element is overloaded, but there are other situations besides overload that can make 408 responses go up 🙈.
This metric is a percentage of total session setup requests. The ISA percentage is calculated like this:
# of ISAs
ISA % = ----------------------------- x 100
Total # of Session RequestsHere's a dialog from [RFC3665] showing the message exchanges for an ineffective session attempt 🦋:
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE | | |
|--------------->|INVITE | |
| 100|--------------->|INVITE |
|<---------------| 100|--------------->|
| |<---------------| |
| | |INVITE |
| | |--------------->|
| | | |
| | |INVITE |
| | |--------------->|
| | | |
| | 408| |
| 408|<---------------| |
|<---------------|ACK | |
| |--------------->| |
|ACK | | |
|--------------->| | |4.9. Session Completion Ratio (SCR)
Okay bestie, a session completion is a SIP dialog that finishes without failing because there was no response from an intended proxy or UA 💜. This is similar to the Call Completion Ratio (CCR) in SIP telephony apps. The output is numerical and it SHOULD be adjusted to show a percentage of successfully completed sessions ✨.
This one's also a percentage, this time of total sessions completed successfully. The SCR percentage is calculated like this:
# of Successfully Completed Sessions
SCR % = --------------------------------------- x 100
Total # of Session RequestsHere's the dialog from [RFC3665] showing the message exchanges needed for a successful session completion 🌈:
UA1 Proxy 1 Proxy 2 UA2
| | | |
|INVITE | | |
|--------------->| | |
| 407| | |
|<---------------| | |
|ACK | | |
|--------------->| | |
|INVITE | | |
|--------------->|INVITE | |
| 100|--------------->|INVITE |
|<---------------| 100|--------------->|
| |<---------------| |
| | | 180|
| | 180 |<---------------|
| 180|<---------------| |
|<---------------| | 200|
| | 200|<---------------|
| 200|<---------------| |
|<---------------| | |
|ACK | | |
|--------------->|ACK | |
| |--------------->|ACK |
| | |--------------->|
| Both Way RTP Media |
|<================================================>|
| | | BYE|
| | BYE|<---------------|
| BYE|<---------------| |
|<---------------| | |
|200 | | |
|--------------->|200 | |
| |--------------->|200 |
| | |--------------->|
| | | |5. Additional Considerations
5.1. Metric Correlations
These metrics can be used to figure out how a domain and/or user is performing 🔍. Here's an example subset of dimensions that give you more granularity for each metric:
To "user"From "user"Bi-direction "user"To "domain"From "domain"Bi-direction "domain"
5.2. Back-to-Back User Agent (B2BUA)
A B2BUA can change how well you can collect these metrics from an end-to-end view 🫣. You have to remember that a B2BUA might act as an originating UAC and terminating UAS, or it might just act as a proxy. Sometimes you need info from both sides of the B2BUA to get the end-to-end picture 💖. Other times the B2BUA is just a proxy, so you can derive the data you need for any of the calculations above 🧚.
5.3. Authorization and Authentication
When a SIP dialog is being set up, different authentication methods might get used 🔐. These add time to what the metrics measure, and how long that takes depends on the method. Failed authentication gets captured by these metrics too, since SIP is what signals whether authorization and/or authentication succeeded or failed 💅. The metrics in Section 3 include the time this takes, even when the method is outside of SIP. This was on purpose, because it really impacts the protocol and the SIP dialogs that come after it 👑.
5.4. Forking
Forking SHOULD be considered when you decide which messages feed the input values for these metrics 🍓. If you used every forked dialog in the calculations, the numbers would get wildly skewed 😵. There are two different points where forking can happen, and each MUST be considered 🎀.
First, forking can happen at a proxy downstream from the UA that's providing the metric input values. That downstream proxy is the one responsible for forking a message. Then this proxy passes along the provisional (e.g., 180) messages it gets from the requests and sends the accepted (e.g., 200) response to the UA 🦋.
Second, if the originating UA or proxy is the one forking the messages, then it MUST parse the message exchanges needed as input for the metrics. For example, it MAY use the first INVITE or set of INVITE messages sent and the first accepted 200 OK. Tags tell you which dialog is distinct from the other 200 OK responses, which get acknowledged and then an immediate BYE gets sent 🌸. The application that captures and/or understands the input values MUST use these tags to tell the dialog requests apart 🫶.
Note that if an INVITE is forked before it reaches its destination, you'll probably get multiple early dialogs, and multiple confirmed dialogs are possible (though unlikely) 👀. When that happens, an SRD measurement should be taken for each dialog that gets created, early or confirmed ✨.
5.5. Data Collection
You can collect the input for these calculations in a bunch of different ways 📊. It can come from call detail records (CDRs) or raw signaling info generated by a proxy or UA. If you're using records, time synchronization MUST be considered between the relevant elements ⏰.
If these metrics get calculated at individual elements (like proxies or endpoints) instead of by a centralized management system, and those elements use different measurement sample sizes, then the metrics reported for the same event can end up looking really different 😭.
The info can also be sent through network management protocols like the Simple Network Management Protocol (SNMP) and through future extensions to the SIP Management Information Base (MIB) modules [RFC4780]. Or it could go through a potential undefined new performance metric event package [RFC3265] that you retrieve with SUBSCRIBE requests 🌈.
Data can be collected for a sample of calls or for all calls, and it can also be derived from test call scenarios. These metrics are super flexible depending on what the application needs 💜.
For consistency in calculating the metrics, elements should expect to reveal event inputs for a centralized management system to use. That system would then calculate the metrics from a varying sample size of inputs received from elements that comply with this specification 🧚.
5.6. Testing Documentation
Sometimes these metrics are used to show how well a specific SIP-based element is performing 🏆. When you use these metrics in a test environment, the environment MUST be accurately documented so you can replicate the output values in future testing and/or validation 📝.
6. Conclusions
This document gives a description of common performance metrics and how they're defined for use with SIP 💅. Using these metrics gives everyone (vendors, service providers, and users) a common viewpoint 🫶. They'll probably get used in production telephony SIP environments to provide input for Key Performance Indicators (KPI) and Service Level Agreement (SLA) indications 👑, but they can also be used to test end-to-end SIP-based service environments ✨.
7. Security Considerations ✨
okay so security here is all about protecting the data you feed into the calculations above 🔒 like, guard that input data like it's your fave hoodie 🧥💖
all the other security stuff? it's covered in RFC 3261 [RFC3261], so go check that out 🌸
bestie, this one's a MUST 🚨
Implementers of these metrics MUST realize that these metrics could be used to describe characteristics of customer and user usage patterns, and privacy should be considered when collecting, transporting, and storing them. 🫶 like, people's usage habits are personal, so treat that data with care 💅
8. Contributors 🎀
these people made substantial contributions to this work, and they're literally iconic 👑
Carol Davids Illinois Institute of Technology
Marian Delkinov Ericsson
Adam Uzelac Global Crossing
Jean-Francois Mule CableLabs
Rich Terpstra Level 3 Communications9. Acknowledgements 🦋
we wanna thank Robert Sparks, John Hearty, and Dean Bayless for reviewing the document and helping clarify some of the trickier parts 🙈 thank you so much!! 🥹 also huge thanks to Dan Romascanu for his insightful comments, and to Vijay Gurbani for agreeing to be the document shepherd 🧚✨
10. References 📚
10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
June 2002.
[RFC3262] Rosenberg, J. and H. Schulzrinne, "Reliability of
Provisional Responses in Session Initiation Protocol
(SIP)", RFC 3262, June 2002.
[RFC3265] Roach, A., "Session Initiation Protocol (SIP)-Specific
Event Notification", RFC 3265, June 2002.
[RFC3665] Johnston, A., Donovan, S., Sparks, R., Cunningham, C.,
and K. Summers, "Session Initiation Protocol (SIP) Basic
Call Flow Examples", BCP 75, RFC 3665, December 2003.
[RFC4780] Lingle, K., Mule, J-F., Maeng, J., and D. Walker,
"Management Information Base for the Session Initiation
Protocol (SIP)", RFC 4780, April 2007.10.2. Informative References
[E.411] ITU-T, "Series E: Overall Network Operation, Telephone
Service, Service Operation and Human Factors", E.411 ,
March 2000.
[E.721] ITU-T, "Series E: Overall Network Operation, Telephone
Service, Service Operation and Human Factors", E.721 ,
May 1999.
[GR-512] Telcordia, "LSSGR: Reliability, Section 12", GR-512-
CORE Issue 2, January 1998.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
"Framework for IP Performance Metrics", RFC 2330,
May 1998.
[RFC5486] Malas, D. and D. Meyer, "Session Peering for Multimedia
Interconnect (SPEERMINT) Terminology", RFC 5486,
March 2009.Authors' Addresses 🗺️
Daryl Malas
CableLabs
858 Coal Creek Circle
Louisville, CO 80027
US
Phone: +1 303 661 3302
EMail: d.malas@cablelabs.com
Al Morton
AT&T Labs
200 Laurel Avenue South
Middletown, NJ 07748
US
Phone: +1 732 420 1571
EMail: acmorton@att.com📦 where this page comes from
- ietf: rfc6076 2011-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms