RFC 8593
Video Traffic Models for RTP Congestion Control Evaluations
Video Traffic Models for RTP Congestion Control Evaluations ๐ฆ
okay bestie, this is RFC 8593 ๐ here's the basic info:
- Authors: X. Zhu (Cisco Systems), S. Mena (Cisco Systems), and Z. Sarker (Ericsson AB) ๐ซถ
- Category: Informational ๐
- Published by: the Internet Engineering Task Force (IETF), ISSN 2070-1721 ๐ธ
- Date: May 2019 ๐๏ธ
- Obsoletes and updates: the header doesn't list anything it obsoletes or updates โจ
Abstract ๐
okay so this doc gives us two reference video traffic models for testing RTP congestion control algorithms ๐. The first one is statistical and basically captures how a live video encoder behaves when the target video rate keeps changing. The second one is trace-driven, which means it copies the real encoded frame sizes from a high-resolution test sequence ๐ผ. Both are built to balance simplicity, repeatability, and authenticity when modeling how a live video source and the congestion control module interact ๐ซถ. At the very end, it explains how both approaches can be combined into a hybrid model ๐.
Status of This Memo ๐ธ
not an Internet Standards Track spec, bestie ๐ this one is published just for information purposes.
It's still a product of the IETF, it reflects the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication ๐. Not every IESG-approved doc is a candidate for any level of Internet Standard, so check Section 2 of RFC 7841 for the deets ๐ง. For the current status, any errata, or a way to send feedback, head to https://www.rfc-editor.org/info/rfc8593 ๐
.
Copyright Notice ๐ฅ
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. ๐
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they're in effect on the publication date. Read those carefully, because they spell out your rights and restrictions ๐. Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License ๐ญ.
Table of Contents ๐ง
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Desired Behavior of a Synthetic Video Traffic Model . . . . . 4
4. Interactions between Synthetic Video Traffic Source and
Other Components at the Sender . . . . . . . . . . . . . . . 5
5. A Statistical Reference Model . . . . . . . . . . . . . . . . 7
5.1. Time-Damped Response to Target-Rate Update . . . . . . . 9
5.2. Temporary Burst and Oscillation during the Transient
Period . . . . . . . . . . . . . . . . . . . . . . . . . 9
5.3. Output-Rate Fluctuation at Steady State . . . . . . . . . 9
5.4. Rate Range Limit Imposed by Video Content . . . . . . . . 10
6. A Trace-Driven Model . . . . . . . . . . . . . . . . . . . . 10
6.1. Choosing the Video Sequence and Generating the Traces . . 11
6.2. Using the Traces in the Synthetic Codec . . . . . . . . . 13
6.2.1. Main Algorithm . . . . . . . . . . . . . . . . . . . 13
6.2.2. Notes to the Main Algorithm . . . . . . . . . . . . . 14
6.3. Varying Frame Rate and Resolution . . . . . . . . . . . . 15
7. Combining the Two Models . . . . . . . . . . . . . . . . . . 16
8. Reference Implementation . . . . . . . . . . . . . . . . . . 17
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17
10. Security Considerations . . . . . . . . . . . . . . . . . . . 17
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 17
11.1. Normative References . . . . . . . . . . . . . . . . . . 17
11.2. Informative References . . . . . . . . . . . . . . . . . 18
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 191. Introduction โจ
okay so when you're evaluating candidate congestion control algorithms for real-time interactive media, you need to account for the traffic patterns a live video encoder generates ๐ฅ. Unlike synthetic sources that can match the rate-changing requests from the congestion control module perfectly, a live encoder can be sluggish about reacting to those changes ๐. Its output rate also usually drifts away from the target rate because of uncertainties in the encoder's rate-control process ๐ฉ. So end-to-end delay and loss for a real-time media flow can get hit even harder by the rate variations the live encoder introduces.
and here's the other side of it ๐ : evaluation results for a candidate RTP congestion control algorithm should mostly reflect how the congestion control module performs, and kinda stay decoupled from the quirks of any specific video codec โจ. It's also desirable that evaluation tests are repeatable and easy to duplicate across different candidate algorithms ๐.
one way to balance all that is to evaluate congestion control algorithms with a synthetic video traffic source model that captures the key behavior of a live video encoder ๐ง. The synthetic model should also have tunable parameters so it can be flexibly adjusted to match the huge variety of real-world live encoder behavior ๐๏ธ. This doc presents two reference models. The first is based on statistical modeling. The second is driven by frame size and interval traces recorded from a real-world encoder ๐. It also talks about the pros and cons of each approach, plus how both can be combined into a hybrid model ๐.
2. Terminology ๐
okay, the BCP 14 key words get a special spot here ๐. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here ๐ซถ.
3. Desired Behavior of a Synthetic Video Traffic Model ๐
A live video encoder uses encoder rate control to hit a target rate ๐ฏ. It does this by tweaking its encoding parameters, like quantization step size, frame rate, and picture resolution, based on its guess about the video content (think motion and scene complexity ๐). But real life gets in the way, and several factors stop the output video rate from perfectly matching the input target rate ๐ค.
Uncertainty in the captured video scene means the output rate usually misses the specified target ๐คท. When there's a big change in target rate, the encoder's output frame sizes can get jittery for a short transient period before the output rate settles on the new target ๐. And even though most frames in a live session are encoded in predictive mode (P-frames in [H264]), the encoder can now and then spit out a large intra-coded frame (I-frame as defined in [H264]), or a frame partially containing intra-coded blocks ๐งฑ. It does that to recover from losses, re-sync with the receiver, or during the transient period of responding to target rate or spatial resolution changes ๐.
So a synthetic video source should have these capabilities ๐:
- Changing bitrate ๐๏ธ: this includes the ability to change frame rate and/or spatial resolution, or to skip frames upon request.
- Fluctuating around the target ใฐ๏ธ: wiggling around the target bitrate that the congestion control module specifies.
- Converging slowly ๐ข: showing a delay in convergence to the target bitrate.
- Repair on demand ๐ฉน: generating intra-coded or repair frames on demand.
There are lots of different ways to build a synthetic video traffic model, but it's desirable that the outcome has a few common characteristics, which are outlined below โจ.
- Low computational complexity ๐ป: the model should be computationally lightweight, because otherwise it totally defeats the whole purpose of being a substitute for a live video encoder ๐ค.
- Temporal pattern similarity โฑ๏ธ: each individual traffic trace the model generates should mimic the temporal pattern of a real video encoder's traces ๐ช.
- Statistical resemblance ๐: the synthetic source should match the real encoder in terms of statistical characteristics, such as the mean, variance, peak, and autocorrelation coefficients of the bitrate. It's also important that this resemblance holds across different time scales, from tens of milliseconds all the way to sub-seconds โณ.
- A wide range of coverage ๐: the model should be easy to configure so it can cover lots of codec behaviors (like fast or slow reaction time in live encoder rate control) and video content variations (like high motion or low motion) ๐ฌ.
These distinct behavior features can be captured via simple statistical modeling or a trace-driven approach. Sections 5 and 6 give an example of each approach, respectively. Section 7 talks about how both models can be combined together ๐ฆ.
4. Interactions between Synthetic Video Traffic Source and Other Components at the Sender โจ
Okay so, Figure 1 shows how the synthetic video traffic source chats with the other components at the sender ๐ฌ like the application, the congestion control module, the media packet transport module, and so on. Both reference models, which we'll get to in Sections 5 and 6, follow this same set of interactions ๐ซถ
The synthetic video source keeps generating a sequence of dummy video frames with varying size and interval ๐ Those dummy frames get processed by the other modules so the video stream can actually go out over the network. Over the whole life of a video session, the synthetic video source is typically going to need to adapt its encoding bitrate, and sometimes its spatial resolution and frame rate too. Main character adaptation energy ๐
In this model, the synthetic video source has a bunch of incoming and outgoing interface calls so it can talk to other modules. Here are some of the incoming interface calls, marked as (a) in Figure 1, that the synthetic video traffic source may accept. This list is not exhaustive, so more interface calls can be added if needed ๐
- Target bitrate R_v: This is the target bitrate request, measured in bits per second (bps) ๐ Usually the congestion control module works out the target bitrate and updates it over time. Depending on which congestion control algorithm is in use, the update requests can either be periodic (e.g., once per second), or on-demand (e.g., only when a drastic bandwidth change over the network is observed). Ngl the on-demand version is giving drama ๐ญ
- Target frame rate FPS: This is the instantaneous frame rate, measured in frames per second at a given time โจ It depends on the native camera-capture frame rate as well as the target/preferred frame rate that the application or user configured ๐
- Target frame resolution XY: This is a 2-dimensional vector that shows the preferred frame resolution in pixels ๐ฆ A bunch of factors decide what resolution gets requested from the synthetic video source over time. Examples include the capturing resolution of the native camera and the display size of the destination screen ๐ฑ The target frame resolution also depends on the current target bitrate R_v, because it does not make sense to pair very low spatial resolutions with very high bitrates, and vice-versa. Mismatch = not it ๐
- Instant frame skipping: This is the request to skip encoding one or several captured video frames, for instance when a drastic decrease in available network bandwidth is detected ๐
- On-demand generation of intra (I) frame: This is the request to encode another I-frame so error propagation at the receiver stops when severe packet losses are observed ๐ง This request typically comes from the error control module. It can be started either by the sender or by the receiver via Full Intra Request (FIR) messages as defined in [RFC5104]. Iconic recovery moment ๐
An example of an outgoing interface call, marked as (b) in Figure 1, is the rate range [R_min, R_max] ๐ธ Here, R_min and R_max are meant to capture the dynamic rate range that the actual live video encoder can generate, given the input video content. This usually depends on the video content complexity and/or the display type. For example, R_max is higher for video content with higher motion complexity or for displays with higher resolution ๐ฌ So these values do not change with R_v, but they can change over time if the content changes ๐ฅ
+-------------+
| | dummy encoded
| Synthetic | video frames
| Video | -------------->
| Source |
| |
+--------+----+
/|\ |
| |
-------------------+ +-------------------->
interface from interface to
other modules (a) other modules (b)
Figure 1: Interaction between Synthetic Video Encoder
and Other Modules at the Sender5. A Statistical Reference Model ๐ฆ
Okay this section is about one simple statistical model of the live video traffic source ๐ Figure 2 sums up the tunable parameters in this model. If you want a more comprehensive survey of popular ways to model video traffic sources, check out [Tanwir2013] ๐
+===========+====================================+================+
| Notation | Parameter Name | Example Value |
+===========+====================================+================+
| R_v | Target bitrate request | 1 Mbps |
+-----------+------------------------------------+----------------+
| FPS | Target frame rate | 30 Hz |
+-----------+------------------------------------+----------------+
| tau_v | Encoder reaction latency | 0.2 s |
+-----------+------------------------------------+----------------+
| K_d | Burst duration of the transient | 8 frames |
| | period | |
+-----------+------------------------------------+----------------+
| K_B | Burst frame size during the | 13.5 KB* |
| | transient period | |
+-----------+------------------------------------+----------------+
| t0 | Reference frame interval 1/FPS | 33 ms |
+-----------+------------------------------------+----------------+
| B0 | Reference frame size R_v/8/FPS | 4.17 KB |
+-----------+------------------------------------+----------------+
| | Scaling parameter of the zero-mean | |
| | Laplacian distribution describing | |
| SCALE_t | deviations in normalized frame | 0.15 |
| | interval (t-t0)/t0 | |
+-----------+------------------------------------+----------------+
| | Scaling parameter of the zero-mean | |
| | Laplacian distribution describing | |
| SCALE_B | deviations in normalized frame | 0.15 |
| | size (B-B0)/B0 | |
+-----------+------------------------------------+----------------+
| R_min | Minimum rate supported by video | 150 kbps |
| | encoder type or content activity | |
+-----------+------------------------------------+----------------+
| R_max | Maximum rate supported by video | 1.5 Mbps |
| | encoder type or content activity | |
+===========+====================================+================+
* Example value of K_B for a video stream encoded at 720p and
30 frames per second using H.264/AVC encoder
Figure 2: List of Tunable Parameters in a Statistical Video Traffic
Source Model5.1. Time-Damped Response to Target-Rate Update โณ
Even though the congestion control module can update its target bitrate request R_v at any time, the statistical model says the encoder only reacts to these changes tau_v seconds after a previous rate transition. Put simply, when the encoder has reacted to a rate-change request at time t, it just ignores all later rate-change requests until time t+tau_v ๐ Very selective, very iconic.
5.2. Temporary Burst and Oscillation during the Transient Period ๐ฅ
The output bitrate R_o during the period [t, t+tau_v] is considered to be in a transient state when the encoder is reacting to abrupt changes in target rate โก From what's been observed in video encoder output, the reaction to a new target bitrate request shows up as high variations in output frame sizes. The model assumes that the overall average output bitrate R_o during this transient period matches the target bitrate R_v. So the occasional burst of large frames gets followed by smaller-than-average encoded frames. It's a whole balancing act ๐คธโโ๏ธ
This temporary burst is described by two parameters:
- burst duration K_d: the number of frames in the burst event, and
- burst frame size K_B: the size of the initial burst frame, which is typically significantly larger than the average frame size at steady state ๐ฆ
Also, these burst parameters can be used to mimic the insertion of a large on-demand I-frame when severe packet losses happen ๐ซ The values of K_d and K_B typically depend on the type of video codec, the spatial and temporal resolution of the encoded stream, and the activity level in the video content ๐๏ธ
5.3. Output-Rate Fluctuation at Steady State ๐
At steady state, the output bitrate R_o is modeled as randomly fluctuating around the target bitrate R_v ๐ซง The output traffic can be described as a combination of two random processes, which track the frame interval t and the output frame size B over time. These are the two major sources of variation in the encoder output. To keep things simple, the deviations of t and B from their reference levels are modeled as independent and identically distributed (i.i.d) random variables following the Laplacian distribution [Papoulis]. More specifically:
- Fluctuations in frame interval: The intervals between adjacent frames have been observed to fluctuate around the reference interval of t0 = 1/FPS. The deviations in normalized frame interval
DELTA_t = (t-t0)/t0can be modeled by a zero-mean Laplacian distribution with scaling parameter SCALE_t. The value of SCALE_t sets the "width" of the Laplacian distribution, and therefore how much the actual frame intervals (t) fluctuate with respect to the reference frame interval t0 ๐ฏ - Fluctuations in frame size: The output-encoded frame sizes also tend to fluctuate around the reference frame size
B0=R_v/8/FPS. Likewise, the deviations in normalized frame sizeDELTA_B = (B-B0)/B0can be modeled by a zero-mean Laplacian distribution with scaling parameter SCALE_B. The value of SCALE_B sets the "width" of this second Laplacian distribution, and so how much the output frame sizes (B) fluctuate with respect to the reference target B0 ๐ง
Both SCALE_t and SCALE_B can be obtained by parameter fitting from empirical data captured for a given video encoder ๐ Example values are listed in Figure 2, based on the empirical data presented in [IETF-Interim].
5.4. Rate Range Limit Imposed by Video Content ๐ฌ
The output bitrate R_o gets further clipped within the dynamic range [R_min, R_max]. In reality, those limits are dictated by the scene and motion complexity of the captured video content ๐ In the proposed statistical model, the application specifies these parameters. Bestie, the app is in charge of the vibe check here ๐ซถ๐
6. A Trace-Driven Model โจ
okay so the second way to model a video source is the trace-driven approach ๐ฌ bestie this one is giving real-life energy. you run an actual live video encoder on a set of chosen raw video sequences, then use the encoder's output traces to build a synthetic video source. the recorded traces naturally wiggle around a given target bitrate request R_v from the congestion control module, and that wiggle is exactly the realistic vibe we want ๐
here's the main plan in five steps:
- Choose one or more representative raw video sequences.
- Encode the sequence(s) using an actual live video encoder. Repeat the process for a number of bitrates. Keep only the sequence of frame sizes for each bitrate.
- Construct a data structure that contains the output of the previous step. The data structure should allow for easy bitrate lookup.
- Upon a target bitrate request R_v from the controller, look up the closest bitrates among those previously stored. Use the frame-size sequences stored for those bitrates to approximate the frame sizes to output.
- The output of the synthetic video traffic source contains "encoded" frames with dummy contents but with realistic sizes.
Section 6.1 covers steps 1-3, and Section 6.2 covers the last two steps (4-5). Section 6.3 is a quick look at how the trace-driven model could be extended to support time-varying frame rate and/or time-varying frame resolution ๐
6.1. Choosing the Video Sequence and Generating the Traces
step one is picking the video sequences really carefully ๐ง they need to be representative of the target use cases for the video traffic model. for the example use case of interactive video conferencing, the recommendation is a "talking head" sequence, like something from a news broadcast or a recording of an actual video call ๐
the length of the chosen sequence is a tradeoff, and it's a bit of a balancing act โ๏ธ if it's too long, the data structures holding the traces get hard to manage. if it's too short, the output frame sizes show an obvious periodic pattern, which leads to biased results when you evaluate congestion control performance. it has been empirically determined that a sequence 2 to 4 minutes in length sufficiently avoids the periodic pattern ๐
so, given the chosen raw video sequence, denoted "S", you can use a live encoder, e.g., some implementation of [H264] or [H265], to produce a set of encoded sequences. as we covered in Section 3, the live encoder's output bitrate can be controlled by tuning three input parameters: quantization step size, frame rate, and picture resolution ๐๏ธ to keep the choice of these parameters simple for a given target rate, you can typically assume a fixed frame rate (e.g., 30 fps) and a fixed resolution (e.g., 720p) when configuring the live encoder. see Section 6.3 for how to relax these assumptions ๐ฆ
with those simplifications in place, the encoder can be configured to start at a constant target bitrate, and then vary the quantization step size (internally via the video encoder rate controller) to hit various externally specified target rates โจ it can also be assumed that the first frame is encoded as an I-frame and the rest are P-frames (see, e.g., [H264] for definitions of I-frames and P-frames). for live encoding, the encoder rate-control algorithm typically doesn't peek at future frames when encoding a given frame, so no cheating allowed ๐ โโ๏ธ
next, we need the bitrate range. given the minimum and maximum bitrates the synthetic codec is supposed to operate at (denoted "R_min" and "R_max", see Section 4), the whole range of target bitrates gets split into n_s steps. that gives an encoding bitrate ladder of (n_s + 1) choices, equally spaced by the step length l = (R_max - R_min)/n_s. the following simple algorithm encodes the raw video sequence:
r = R_min
while r <= R_max do
Traces[r] = encode_sequence(S, r, e)
r = r + lthe function encode_sequence takes three inputs: a raw video sequence (S), a constant target rate (r), and an encoder rate-control algorithm (e). it returns a vector with the sizes of the frames in the order they were encoded ๐ฆ the output vector gets stored in a map structure called "Traces", whose keys are bitrates and whose values are vectors of frame sizes ๐๏ธ
the value you pick for the number of bitrate steps n_s matters a lot, because it decides how many vectors of frame sizes end up in the map Traces ๐ the smallest value you can choose for n_s is 1, and the biggest depends on how much memory you have for holding Traces. a reasonable value for n_s is one that gives steps of length l = 200 kbps. Section 6.2.2 digs further into the choice of step length l ๐
last thing: as mentioned in earlier sections, R_min and R_max might be modified after the initial sequences are encoded. so, for clarity, we'll call the bitrate range of the trace file [Rf_min, Rf_max] from here on out ๐ธ the algorithm in Section 6.2.1 also handles the cases where the current target bitrate is less than Rf_min or greater than Rf_max, so you're covered either way ๐ซถ
6.2. Using the Traces in the Synthetic Codec
the main idea behind the trace-driven synthetic codec is that it mimics the rate-adaptation behavior of a real live codec when the congestion control module dynamically updates the target bitrate request R_v ๐ฅ it does this by switching to a different frame-size vector stored in the map Traces whenever it's needed. iconic behavior, honestly ๐
6.2.1. Main Algorithm
the main algorithm for rate adaptation in the synthetic codec keeps track of two variables: r_current and t_current ๐ง
- The variable r_current points to one of the keys of the map Traces. Whenever R_v changes, which typically happens because the congestion controller notices the network conditions have changed, r_current gets updated based on R_v like this:
R_ref = min (Rf_max, max(Rf_min, R_v))
r_current = r
such that
(r in keys(Traces) and
r <= R_ref and
(not(exists) r' in keys(Traces) such that r <r'<= R_ref))- The variable t_current is an index into the frame-size vector stored in Traces[r_current]. It gets updated every time a new frame is due ๐ฌ all the vectors stored in Traces are assumed to have the same size, called "size_traces". the update of t_current follows this equation:
if t_current < SkipFrames then
t_current = t_current + 1
else
t_current = ((t_current + 1 - SkipFrames)
% (size_traces-SkipFrames)) + SkipFrameswhere the operator "%" means modulo, and SkipFrames is a predefined constant for the number of frames to skip at the beginning of the frame-size vectors after t_current has wrapped around. the whole point of SkipFrames is to avoid the effect of periodically sending a big I-frame followed by several smaller-than-average P-frames ๐ a typical value of SkipFrames is 20, though it could be set to 0 if you're interested in studying the effect of sending I-frames periodically ๐
the initial value of r_current is set to R_min, and the initial value of t_current is set to 0 ๐
when a new frame is due, its size can be calculated using one of the three cases below:
a) Rf_min <= R_v < Rf_max: The output frame size is calculated via linear interpolation of the frame sizes in Traces[r_current] and Traces[r_current + l]. the interpolation goes like this:
size_lo = Traces[r_current][t_current]
size_hi = Traces[r_current + l][t_current]
distance_lo = (R_v - r_current) / l
framesize = size_hi*distance_lo + size_lo*(1-distance_lo)b) R_v < Rf_min: The output frame size is calculated via scaling with respect to the lowest bitrate Rf_min in the trace file, like so:
w = R_v / Rf_min
framesize = max(fs_min, factor * Traces[Rf_min][t_current])c) R_v >= Rf_max: The output frame size is calculated by scaling with respect to the highest bitrate Rf_max in the trace file, like so:
w = R_v / Rf_max
framesize = min(fs_max, w * Traces[Rf_max][t_current])in cases b) and c), floating-point arithmetic is used to compute the scaling factor "w" ๐งฎ the resulting instantaneous frame size (framesize) is then clipped to a reasonable range between fs_min (e.g., 10 bytes) and fs_max (e.g., 1 MB) so nothing gets too tiny or too huge ๐
6.2.2. Notes to the Main Algorithm
the main algorithm above can be extended to mimic some extra typical behaviors of a live video encoder ๐ฆ here are two examples:
- I-frames on demand: The synthetic codec can be extended to simulate sending I-frames on demand, e.g., as a reaction to losses. to do this, the codec's incoming interface (see (a) in Figure 1) gets a new function to request a new I-frame. when that function is called, t_current is reset to 0 ๐ง
- Variable step length l between R_min and R_max: In the main algorithm, the step length l is fixed to keep things easy to follow. but if the range [R_min, R_max] is very wide, you can also define a set of intermediate encoding rates with variable step length ๐ธ the reasoning is that the difference between 400 and 600 kbps as a target bitrate is way more significant than the difference between 4400 kbps and 4600 kbps ๐ for example, you could define steps of 200 kbps under 1 Mbps, then steps of 300 kbps between 1 Mbps and 2 Mbps, then 400 kbps between 2 Mbps and 3 Mbps, and so on ๐
6.3. Varying Frame Rate and Resolution โจ
okay so the trace-driven synthetic codec in this section is pretty simple bc it fixes the frame rate and frame resolution ๐ง. but it can totally be extended to handle variable frame rate and/or variable spatial resolution, fr fr ๐
when the encoded picture quality at a given bitrate is low, you can lower the frame rate (if the video is in low motion) or the spatial resolution to boost the quality of experience (QoE) of the whole encoded video ๐ฅบ. flip side: if the target bitrate climbs to the point where single frames don't look any better, you could bump up the spatial resolution or the frame rate (super useful when the video is in high motion) ๐ฆ
people have proposed a bunch of ways to pick the best mix of encoder-quantization step size, frame rate, and spatial resolution over time to maximize quality in live video codecs [Ozer2011] [Hu2012] ๐ค. future work might extend the trace-driven codec to handle variable frame rate and/or resolution โจ
from the congestion control point of view, changing spatial resolution usually means generating a new intra-coded frame, which causes a temporary burst in the output traffic ๐ฅ. frame-rate changes are sneakier: dropping from high to low gives you sparsely spaced, larger encoded packets instead of lots of densely spaced smaller ones ๐ธ. that difference in traffic profile can still mess with congestion control performance, especially when outgoing packets are not paced by the media transport module ๐. investigating varying frame rate and resolution is left for future work ๐ซถ
7. Combining the Two Models ๐
so the statistical and trace-driven models each have their own pros and cons, but both are pretty simple to implement ๐. the hard part is fitting the parameters of a statistical model to actual encoder output data, which takes way more effort ๐ญ. grabbing encoded frame-size data for a trace-driven model is super straightforward though ๐. once validated, the statistical model is really flexible: you just vary its parameters to mimic a wide range of encoder and content behaviors โจ. a trace-driven model, by definition, relies on extra data-collection work every time you want to accommodate new codecs or video content ๐
generally, the trace-driven model is more realistic for mimicking the ongoing steady-state behavior of a video traffic source fluctuating around a constant target rate ๐ฆ. the statistical model is the versatile one for transient stuff, like sudden rate changes โก. you can also combine both into a hybrid model, where the steady state is driven by traces and the transient state is driven by the statistical model ๐
transient +---------------+
state | Generate next |
+------>| K_d transient |
+-----------------+ / | frames |
R_v | Compare against | / +---------------+
------>| previous |/
| target bitrate |\
+-----------------+ \ +---------------+
\ | Generate next |
+------>| frame from |
steady | trace |
state +---------------+Figure 3: A Hybrid Video Traffic Model
as Figure 3 shows, the video traffic model operates in a transient state if the requested target rate R_v is substantially different from the previous target; otherwise, it operates in a steady state ๐ง . during the transient state, a total of K_d frames are generated by the statistical model, giving you one (1) big burst frame with size K_B followed by K_d-1 smaller frames ๐ฅ. in steady state, the model just generates a frame according to the trace-driven model for the given target rate, while modulating the frame interval per the distribution from the statistical model ๐. one example criterion for deciding transient vs. steady state is whether the rate change exceeds 10% of the previous target rate ๐. last but not least, since this model follows transient-state behavior from the statistical model, after a substantial rate change it follows the time-damping mechanism defined in Section 5.1, governed by parameter tau_v โณ
8. Reference Implementation ๐ง
the statistical, trace-driven, and hybrid models in this document have all been implemented as a stand-alone, platform-independent synthetic traffic source module ๐. it can be easily plugged into network simulation platforms like [ns-2] and [ns-3], as well as testbeds using a real network ๐. the stand-alone traffic source module is open source and lives at [Syncodecs] ๐ป
9. IANA Considerations ๐
this document has no IANA actions ๐คทโโ๏ธ
10. Security Considerations ๐จ
okay, serious moment: the synthetic video traffic models in this document do not impose any security threats ๐. they're designed to mimic realistic traffic patterns so you can evaluate candidate RTP-based congestion control algorithms and make sure the network stays stable ๐ก๏ธ. It is RECOMMENDED that candidate algorithms be tested using the video traffic models presented in this document before wide deployment over the Internet ๐. if the generated synthetic traffic flows are sent over the Internet, they also need to be congestion controlled, no exceptions ๐ค๐
11. References ๐
11.1. Normative References
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997,
https://www.rfc-editor.org/info/rfc2119. - [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
https://www.rfc-editor.org/info/rfc8174.
11.2. Informative References
- [H264] ITU-T, "Advanced video coding for generic audiovisual services", Recommendation H.264, April 2017,
https://www.itu.int/rec/T-REC-H.264. - [H265] ITU-T, "High efficiency video coding", Recommendation H.265, February 2018,
https://www.itu.int/rec/T-REC-H.265. - [Hu2012] Hu, H., Ma, Z., and Y. Wang, "Optimization of Spatial, Temporal and Amplitude Resolution for Rate-Constrained Video Coding and Scalable Video Adaptation", Proc. 19th IEEE International Conference on Image Processing (ICIP), DOI 10.1109/ICIP.2012.6466960, September 2012.
- [IETF-Interim] Zhu, X., Mena, S., and Z. Sarker, "Update on RMCAT Video Traffic Model: Trace Analysis and Model Update", IETF RMCAT Virtual Interim, April 2017,
https://www.ietf.org/proceedings/interim-2017-rmcat-01/slides/slides-interim-2017-rmcat-01-sessa-update-on-video-traffic-model-draft-00.pdf. - [ns-2] "The Network Simulator - ns-2", December 2015,
https://nsnam.sourceforge.net/wiki/index.php/User_Information. - [ns-3] "NS-3 Network Simulator",
https://www.nsnam.org/. - [Ozer2011] Ozer, J., "Video Compression for Flash, Apple Devices and HTML5", Galax: Doceo Publishing, ISBN-13: 978-0976259503, 2011.
- [Papoulis] Papoulis, A. and S. Pillai, "Probability, Random Variables and Stochastic Processes", London: McGraw-Hill Europe, ISBN-13: 978-0071226615, 2002.
- [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, "Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104, February 2008,
https://www.rfc-editor.org/info/rfc5104. - [Syncodecs] "Syncodecs: Synthetic codecs for evaluation of RMCAT work", commit a92d6c8, May 2018,
https://github.com/cisco/syncodecs. - [Tanwir2013] Tanwir, S. and H. Perros, "A Survey of VBR Video Traffic Models", IEEE Communications Surveys and Tutorials, Volume 15, Issue 4, p. 1778-1802, DOI 10.1109/SURV.2013.010413.00071, January 2013.
Authors' Addresses ๐
Xiaoqing Zhu
Cisco Systems
12515 Research Blvd., Building 4
Austin, TX 78759
United States of America
Email: xiaoqzhu@cisco.com
Sergio Mena
Cisco Systems
EPFL, Quartier de l'Innovation, Batiment E
Ecublens, Vaud 1015
Switzerland
Email: semena@cisco.com
Zaheduzzaman Sarker
Ericsson AB
Torshamnsgatan 23
Stockholm, SE 164 83
Sweden
Phone: +46 10 717 37 43
Email: zaheduzzaman.sarker@ericsson.com๐ฆ where this page comes from
- ietf: rfc8593 2019-05
- ๐ girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms