RFC 6236
Negotiation of Generic Image Attributes in the Session Description Protocol (SDP)
RFC 6236: Negotiation of Generic Image Attributes in the Session Description Protocol (SDP) 🎀
Okay bestie, here's the deal 💅 RFC 6236 is an Internet Standards Track document from May 2011 (ISSN 2070-1721). I. Johansson (Ericsson AB) and K. Jung (Samsung Electronics Co., Ltd.) wrote it. It doesn't list anything it obsoletes or updates, so it's a standalone glow-up. ✨
Abstract 💖
Okay so this doc proposes a brand-new generic session setup attribute so two devices can negotiate image attributes like image size 📐 The main use case is a low-end handheld terminal that wants to show video without rescaling the image, which can eat up a ton of memory and processing power 😭 It also helps keep the bitrate for video optimal, because only the image size the receiver actually wants gets sent. Efficient and iconic 🦋
Status of This Memo 🌸
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741.
Info about the current status of this document, any errata, and how to give feedback on it lives at http://www.rfc-editor.org/info/rfc6236 🔗
Copyright Notice 🧚
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Table of Contents 📚
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Requirements . . . . . . . . . . . . . . . . . . . . . . . 4
2. Conventions Used in This Document . . . . . . . . . . . . . . 5
3. Specification of the 'imageattr' SDP Attribute . . . . . . . . 5
3.1. Attribute Syntax . . . . . . . . . . . . . . . . . . . . . 5
3.1.1. Overall View of Syntax . . . . . . . . . . . . . . . . 5
3.2. Considerations . . . . . . . . . . . . . . . . . . . . . . 11
3.2.1. No imageattr in First Offer . . . . . . . . . . . . . 11
3.2.2. Different Payload Type Numbers in Offer and Answer . . 11
3.2.3. Asymmetry . . . . . . . . . . . . . . . . . . . . . . 12
3.2.4. sendonly and recvonly . . . . . . . . . . . . . . . . 12
3.2.5. Sample Aspect Ratio . . . . . . . . . . . . . . . . . 13
3.2.6. SDPCapNeg Support . . . . . . . . . . . . . . . . . . 13
3.2.7. Interaction with Codec Parameters . . . . . . . . . . 14
3.2.8. Change of Display in Middle of Session . . . . . . . . 16
3.2.9. Use with Layered Codecs . . . . . . . . . . . . . . . 16
3.2.10. Addition of Parameters . . . . . . . . . . . . . . . . 16
4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1. A High-Level Example . . . . . . . . . . . . . . . . . . . 16
4.2. Detailed Examples . . . . . . . . . . . . . . . . . . . . 17
4.2.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . 17
4.2.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . 18
4.2.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . 19
4.2.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . 20
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
6. Security Considerations . . . . . . . . . . . . . . . . . . . 21
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 21
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 22
8.1. Normative References . . . . . . . . . . . . . . . . . . . 22
8.2. Informative References . . . . . . . . . . . . . . . . . . 221. Introduction ✨
So this doc is all about a new SDP attribute that lets devices negotiate image attributes, like image size 📺 "Image size" has a specific definition here, because it can be different from the physical screen size of something like a handheld 📱 For example, you might want to show a video on just part of the screen and leave room for menus and other info. Main character behavior 💅
Letting the image size get negotiated gets you a bunch of perks 🎀
- Less image distortion: Rescaling images adds extra distortion, which can be avoided (at least on the receiver side) if the image size can be negotiated.
- Reduced receiver complexity: Image rescaling can be quite computation intensive. For low-end devices, this can be a problem.
- Optimal quality for the given bitrate: The sender does not need to encode an entire CIF (352x288) image if only an image size of 288x256 is displayed on the receiver screen.
- Memory requirement: The receiver device will know the size of the image and can then allocate memory accordingly.
- Optimal aspect ratio: The indication of the supported image sizes and aspect ratio allows the receiver to select the most appropriate combination based on its rescaling capabilities and the desired rendering. For example, if a sender proposes three resolutions in its SDP offer (100x200, 200x100, and 100x100) with
sar=1.0(1:1) etc., then the receiver can select the option that fits the receiver screen best.
In cases where rescaling is not implemented (for example, rescaling is not mandatory to implement in H.264 [H.264]), indicating the image attributes can still give you optimal bandwidth use 🦋 The attribute gives the encoder a better idea of which image size is preferred anyway, so it helps avoid wasting bandwidth by encoding at an unnecessarily large resolution. Iconic efficiency 🫶
For implementers thinking about rescaling, there are several reasons to do it on the sender side 👑
- Rescaling on the sender/encoder side is likely to be easier to do, as the camera-related software/hardware already has the necessary functionality for zooming/cropping/trimming/sharpening the video signal. Plus, rescaling is generally done in RGB or YUV domains and should not depend on the codecs used.
- The encoder may be able to encode in a number of formats but may not know which format to choose, because without the image attribute it does not know the receiver's performance or preference.
- The quality drop due to digital domain rescaling using interpolation is likely to be lower if it is done before the video encoding rather than after the decoding, especially when low bitrate video coding is used.
- If low-complexity rescaling operations such as simple cropping must be performed, the benefit of having this functionality on the sender side is that it is then possible to present a miniature "what you send" image on the display to help the user frame the image correctly. 🎬
Several of the existing standards ([H.263], [H.264], and [MPEG-4]) have support for different resolutions at different framerates. The purpose of this document is to provide a generic mechanism, which is targeted mainly at negotiating the image size. To make it more general, though, the attribute is named 'imageattr' 🌈
This document is limited to point-to-point unicast communication scenarios. The attribute may be used in centralized conferencing scenarios as well, but due to the abundance of configuration options, it may then be difficult to come up with a configuration that fits all parties 🙈
1.1. Requirements 🍓
The design of the image attribute is based on the following requirements, which are listed only for informational purposes:
REQ-1: Support the indication of one or more set(s) of image attributes that the SDP endpoint wishes to receive or send. Each image attribute set must include a specific image size.
REQ-2: Support setup/negotiation of image attributes, meaning that each side in the Offer/Answer should be able to negotiate the image attributes it prefers to send and receive.
REQ-3: Interoperate with codec-specific parameters such as sprop-parameter-sets in [H.264] or config in [MPEG-4].
REQ-4: Make the attribute generic with as few codec specific details/tricks as possible in order to be codec agnostic.
Besides the above-mentioned requirements, the requirement below may be applicable.
OPT-1: The image attribute should support the description of image-related attributes for various types of media, including video, pictures, images, etc.
2. Conventions Used in This Document 📖
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
3. Specification of the 'imageattr' SDP Attribute 💜
This section defines the SDP image attribute 'imageattr', which can be used in an SDP Offer/Answer exchange to indicate various image attribute parameters. In this document, we define the following image attribute parameters: image resolution, sample aspect ratio (sar), allowed range in picture aspect ratio (par), and the preference of a given parameter set over another (q). The attribute is extensible, and guidelines for defining additional parameters are provided in Section 3.2.10.
3.1. Attribute Syntax 🧩
In this section, the syntax of the 'imageattr' attribute is described. The 'imageattr' attribute is a media-level attribute. The section is split into two parts: the first gives an overall view of the syntax, and the second describes how the syntax is used.
3.1.1. Overall View of Syntax ✨
okay so bestie, this is the grammar for the image attribute, written in ABNF [RFC5234] 🧚 It's the whole blueprint, so read it slowly.
image-attr = "imageattr:" PT 1*2( 1*WSP ( "send" / "recv" )
1*WSP attr-list )
PT = 1*DIGIT / "*"
attr-list = ( set *(1*WSP set) ) / "*"
; WSP and DIGIT defined in [RFC5234]
set= "[" "x=" xyrange "," "y=" xyrange *( "," key-value ) "]"
; x is the horizontal image size range (pixel count)
; y is the vertical image size range (pixel count)
key-value = ( "sar=" srange )
/ ( "par=" prange )
/ ( "q=" qvalue )
; Key-value MAY be extended with other keyword
; parameters.
; At most, one instance each of sar, par, or q
; is allowed in a set.
;
; sar (sample aspect ratio) is the sample aspect ratio
; associated with the set (optional, MAY be ignored)
; par (picture aspect ratio) is the allowed
; ratio between the display's x and y physical
; size (optional)
; q (optional, range [0.0..1.0], default value 0.5)
; is the preference for the given set,
; a higher value means a higher preference
onetonine = "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" / "9"
; Digit between 1 and 9
xyvalue = onetonine *5DIGIT
; Digit between 1 and 9 that is
; followed by 0 to 5 other digits
step = xyvalue
xyrange = ( "[" xyvalue ":" [ step ":" ] xyvalue "]" )
; Range between a lower and an upper value
; with an optional step, default step = 1
; The rightmost occurrence of xyvalue MUST have a
; higher value than the leftmost occurrence.
/ ( "[" xyvalue 1*( "," xyvalue ) "]" )
; Discrete values separated by ','
/ ( xyvalue )
; A single value
spvalue = ( "0" "." onetonine *3DIGIT )
; Values between 0.1000 and 0.9999
/ ( onetonine "." 1*4DIGIT )
; Values between 1.0000 and 9.9999
srange = ( "[" spvalue 1*( "," spvalue ) "]" )
; Discrete values separated by ','.
; Each occurrence of spvalue MUST be
; greater than the previous occurrence.
/ ( "[" spvalue "-" spvalue "]" )
; Range between a lower and an upper level (inclusive)
; The second occurrence of spvalue MUST have a higher
; value than the first
/ ( spvalue )
; A single value
prange = ( "[" spvalue "-" spvalue "]" )
; Range between a lower and an upper level (inclusive)
; The second occurrence of spvalue MUST have a higher
; value than the first
qvalue = ( "0" "." 1*2DIGIT )
/ ( "1" "." 1*2("0") )
; Values between 0.00 and 1.00- The attribute typically contains a "send" and a "recv" keyword. These specify the preferences for the media once the session is set up, in the send and receive direction respectively from the point of view of the sender of the session description. One of the keywords ("send" or "recv") MAY be omitted; see Section 3.2.4 and Section 3.2.2 for a description of cases when this may be appropriate.
- The "send" keyword and corresponding attribute list (attr-list) MUST NOT occur more than once per image attribute.
- The "recv" keyword and corresponding attribute list (attr-list) MUST NOT occur more than once per image attribute.
- PT is the payload type number; it MAY be set to "*" (wild card) to indicate that the attribute applies to all payload types in the media description.
- For sendrecv streams, both of the send and recv directions SHOULD be present in the SDP.
- For inactive streams it is RECOMMENDED that both of the send and recv directions are present in the SDP.
3.1.1.1. Parameter Rules
The following rules apply for the parameters.
Payload type number (PT): The image attribute is bound to a specific codec by means of the payload type number. A wild card (*) can be specified for the payload type number to indicate that it applies to all payload types in the media description. Several image attributes can be defined, for instance for different video codec alternatives. This however requires that the payload type numbers differ. Note that the attribute is associated to the codec(s), for instance an SDP offer may specify payload type number 101 while the answer may specify 102, this may make it troublesome to specify a payload type number with the 'imageattr' attribute. See Section 3.2.2 for a discussion and recommendation how this is solved.
Preference (q): The preference for each set is 0.5 by default; setting the optional q parameter to another value makes it possible to set different preferences for the sets. A higher value gives a higher preference for the given set.
sar: The sar (storage aspect ratio) parameter specifies the sample aspect ratio associated to the given range of x and y values. The sar parameter is defined as dx/dy where dx and dy are the physical size of the pixels. Square pixels gives a sar=1.0. The parameter sar MAY be expressed as a range or as a single value.
If this parameter is not present, a default sar value of 1.0 is assumed.
The interpretation of sar differs between the send and the receive directions.
- In the send direction, sar defines a specific sample aspect ratio associated to a given x and y image size (range).
- In the recv direction, sar expresses that the receiver of the given medium prefers to receive a given x and y resolution with a given sample aspect ratio.
See Section 3.2.5 for a more detailed discussion.
The sar parameter will likely not solve all the issues that are related to different sample aspect ratios, but it can help to solve them and reduce aspect ratio distortion.
The response MUST NOT include a sar parameter if there is no acceptable value given. The reason for this is that if the response includes a sar parameter it is interpreted as "sar parameter accepted", while removal of the sar parameter is treated as "sar parameter not accepted". For this reason, it is safer to remove an unacceptable sar parameter altogether.
par: The par (width/height = x/y ratio) parameter indicates a range of allowed ratios between x and y physical size (picture aspect ratio). This is used to limit the number of x and y image size combinations; par is given as
par=[ratio_min-ratio_max]where ratio_min and ratio_max are the min and max allowed picture aspect ratios.
If sar and the sample aspect ratio that the receiver actually uses in the display are the same (or close), the relation between the x and y pixel resolution and the physical size of the image is straightforward. If however sar differs from the sample aspect ratio of the receiver display, this must be taken into consideration when the x and y pixel resolution alternatives are sorted out. See Section 4.2.4 for an example of this.
3.1.1.2. Offer/Answer Rules
In accordance with [RFC3264], offer/answer exchange of the image attribute is as follows.
- Offerer sending the offer:
- The offerer must be able to support the image attributes that it offers, unless the offerer has expressed a wild card (*) in the attribute list.
- It is recommended that a device that sees no reason to use the image attribute includes the attribute with wild cards (*) in the attribute lists anyway for the send and recv directions. An example of this looks like:
a=imageattr:97 send * recv *This gives the answerer the possibility of expressing its preferences. The use of wild cards introduces a risk that the message size can increase in an uncontrolled way. To reduce this risk, these wild cards SHOULD only be replaced by an as small set as possible.
- Answerer receiving the offer and sending the answer:
- The answerer may choose to keep the image attribute but is not required to do so.
- The answerer may, for its receive and send direction, include one or more entries that it can support from the set of entries proposed in the offer.
- The answerer may also, for its receive and send direction, replace the entries with a complete new set of entries different from the original proposed by the offerer. The implementor of this feature should however be aware that this may cause extra offer/answer exchanges.
- The answerer may also remove its send direction completely if it is deemed that it cannot support any of the proposed entries.
- The answerer should not include an image attribute in the answer if it was not present in the offer.
- Offerer receiving the answer:
- If the image attribute is not included in the SDP answer the offerer SHOULD continue to process the answer as if this mechanism had not been offered.
- If the image attribute is included in the SDP answer but none of the entries are usable or acceptable, the offerer MUST resort to other methods to determine the appropriate image size. In this case, the offerer must also issue a new offer/answer without the image attribute to avoid misunderstandings between the offerer and answerer. This will avoid the risk of infinite negotiations.
3.2. Considerations
3.2.1. No imageattr in First Offer
When the initial offer does not contain the 'imageattr' attribute, the rules in Section 3.1.1.2 require the attribute to be absent in the answer. The reasons for this are:
- The offerer of the initial SDP is not likely to understand the image attribute if it did not include it in the offer, bearing in mind that Section 3.1.1 recommends that the offerer provide the attribute with wild carded parameters if it has no preference.
- Inclusion of the image attribute in the answer may come in conflict with the rules in Section 3.1.1.2, especially the rules that apply to "offerer receiving the answer".
For the above reasons, it is RECOMMENDED that a device that sees no reason to use the image attribute includes the attribute with wild cards (*) in the attribute lists anyway for the send and recv directions.
3.2.2. Different Payload Type Numbers in Offer and Answer 🎀
okay so sometimes the answer is allowed to use a different payload type number than the offer. Like, the offer might say
m=video 49154 RTP/AVP 99and then the answer comes back with
m=video 49154 RTP/AVP 100🙈 Now if the image attribute in the offer says payload type 99, that attribute has no meaning in the answerer's receive direction, because the m-line says the media payload type is 100. Chaos, bestie! 💀
There are a few ways to fix this:
- Use a wild card
*as the payload type number in the image attribute in the offer SDP. The answer SDP uses the wild card too. The drawback is that this attribute then applies to all payload type numbers in the media description. - Use a wild card
*as the payload type number in the image attribute in the answer SDP. The offer SDP can contain a defined payload type number in the image attribute, but the answer SDP swaps it for a wild card. The drawback is similar to the one above. - Split the image attribute in two parts in the SDP answer. For example, the offer SDP (only the parts we care about here) looks like this:
m=video 49154 RTP/AVP 99
a=imageattr:99 send ... recv ...and the answer SDP looks like this:
m=video 49154 RTP/AVP 100
a=imageattr:99 send ...
a=imageattr:100 recv ...This last option has no drawbacks 🫶 and it even lets you specify different image attributes if more than one payload type is in the offer SDP. Slay.
Of the alternatives above, the last one MUST be used because it is the safest. The other alternatives MUST NOT be used. 👑
3.2.3. Asymmetry 🦋
The image attribute supports asymmetry, but there are some limitations. The big one is that the codec can only support up to a certain maximum resolution for a given profile level. 😭
For example, H.264 [H.264] with profile level 1.2 doesn't support anything bigger than 352x288 (CIF). The offer/answer rules imply that the same profile level has to be used in both directions. So in an asymmetric scenario where Alice wants an image size of 580x360 and Bob wants 150x120, profile level 2.2 is needed in both directions, even though profile level 1 would have been enough in one direction. Not fair, but that's the vibe.
Right now, the only solution is to specify two unidirectional media descriptions. Note however that the asymmetry issue for the H.264 codec is solved by the level-asymmetry-allowed parameter in [RFC6184]. ✨
3.2.4. sendonly and recvonly 🌸
If the directional attributes a=sendonly or a=recvonly are given for a medium, there's obviously no need to specify the image attribute for both directions. So one of the directions can be left out. 💅 That said, it may be good to do the image attribute negotiation in both directions, just in case the session gets updated for media in both directions later on.
3.2.5. Sample Aspect Ratio 📐
The relationship between the sar parameter and the x and y pixel resolution deserves some extra discussion, bestie. Let's look at the offer from Alice to Bob (we'll set the recv direction aside for now):
a=imageattr:97 send [x=720,y=576,sar=1.1]If the receiver's display has square pixels, the 720x576 image would need to be rescaled to, for example, 792x576 or 720x524 to keep a correct image aspect ratio. In practice that means the rescaling would have to happen on the receiver side, which is contrary to the spirit of this document. 🙅♀️
To avoid this problem, Alice can specify a range of values for the sar parameter, like this:
a=imageattr:97 send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]]This means Alice can encode with any of the mentioned sample aspect ratios and leaves Bob to decide which one he prefers. Main character energy 🧚
3.2.6. SDPCapNeg Support 🍓
The image attribute can be used within the SDP Capability Negotiation [RFC5939] framework, and its use is then specified using the a=acap parameter. An example is:
a=acap:1 imageattr:97 send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]]For use with the SDP Media Capability Negotiation extension [SDPMedCapNeg], it's no longer possible to specify payload type numbers. So you can use the parameter substitution rule instead. Here's an example:
...
a=mcap:1 video H264/90000
a=acap:1 imageattr:%1% send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]]
...where %1% maps to media capability number 1.
You can also use the a=mscap attribute, like in the example below:
...
a=mcap:1 video H264/90000
a=mscap:1 imageattr send [x=720,y=576,sar=[0.91,1.0,1.09,1.45]]
...3.2.7. Interaction with Codec Parameters 🔥
Most codecs already have some kind of indication of, for example, the image size at session setup. So measures must be taken to avoid conflicts between the image attribute and that existing info. Conflicts are not it, periodt.
The subsections below describe the most well-known codecs and how they define image-size related information. Section 3.2.7.4 outlines a few possible solutions, but this document doesn't make a recommendation for any of them. 🙈
3.2.7.1. H.263 🌈
The payload format for H.263 [H.263] is described in [RFC4629].
H.263 defines (on the fmtp line) a list of image sizes and their maximum frame rates (profiles) that the offerer can receive. The answerer is not allowed to modify this list and must reject a payload type that contains an unsupported profile. The CUSTOM profile may be used for image size negotiation, but support for asymmetry requires specifying two unidirectional media descriptions using the sendonly/recvonly attributes. ✨
3.2.7.2. H.264 🦋
The payload format for H.264 [H.264] is described in [RFC6184].
H.264 defines information related to image size in the fmtp line by means of sprop-parameter-sets. According to the specification, several sprop-parameter-sets may be defined for one payload type. The sprop-parameter-sets describe the image size (+ more) that the offerer sends in the stream, and they don't need to be complete. This means that sprop-parameter-sets does not represent any negotiation, and the answer is not allowed to change them.
This configuration may be changed later inband if, for instance, image sizes need to be changed or added. 💖
3.2.7.3. MPEG-4 🌸
The payload format for MPEG-4 [MPEG-4] is described in [RFC3016].
MPEG-4 defines a config parameter on the fmtp line, which is a hexadecimal representation of the MPEG-4 visual configuration information. This configuration does not represent any negotiation, and the answer is not allowed to change the parameter.
It is not possible to change the configuration using inband signaling. 😩
3.2.7.4. Possible Solutions 💡
The subsections above clearly show that this kind of information must be aligned well with the image attribute to avoid conflicts. There are a number of possible solutions, listed below without any preference:
- Ignore payload format parameters: This may not work well in the presence of bad channel conditions, especially at the beginning of a session. Moreover, this is not a good option for MPEG-4.
- Second session-wide offer/answer round: In the second offer/answer, the parameters specific to the codec payload format are defined based on the outcome of the 'imageattr' negotiation. The drawback is that setup of the entire session (including audio) may be delayed considerably, especially since the 'imageattr' negotiation can already cost up to two offer/answer rounds. Also, the conflict between the 'imageattr' negotiation and the payload-format-specific parameters is still present after the first offer/answer round, and a fuzzy/buggy implementation may start media before the second offer/answer is completed, with unwanted results.
- Second session-wide offer/answer round only for video: This is similar to the alternative above, with the exception that setup time for audio is not increased. Moreover, the port number for video is set to 0 during the first offer/answer round to avoid the flow of media.
This has the effect that video will blend in some time after the audio is started (up to 2 seconds of delay). This alternative is likely the most clean-cut and failsafe. The drawback is that, since the port number in the first offer is always zero, media startup will always be delayed, even though it would in fact have been possible to start media after the first offer/answer round.
Note that according to [RFC3264], a port number of zero means the whole media line is rejected, which means a new offer for the same port number should be treated as a completely new stream and not as an update. The safest way to solve this problem is to use preconditions, but this is outside the scope of this document. 🫶
3.2.8. Change of Display in Middle of Session 📱
A very likely scenario is that a user switches to another phone during a video telephony call, or plugs a cellphone into an external monitor. In both cases, it's very likely that a renegotiation gets initiated using the SIP-REFER [RFC3515] or SIP-UPDATE [RFC3311] methods. It is RECOMMENDED to negotiate the image size during this renegotiation. ✨
3.2.9. Use with Layered Codecs 🧅
The image attribute is a media-level attribute, so its use with layered codecs causes some concern. If the layers are transported in different RTP streams, the layers are specified on different media descriptions, and the relation is specified using the grouping framework [RFC5888] and the depend attribute [RFC5583]. Since it's not possible to specify only one image attribute for several media descriptions, the solution is either to specify the same image attribute for each media description, or to specify the image attribute only for the base layer. 💜
3.2.10. Addition of Parameters 🌈
The image attribute allows for the addition of parameters in the future. To make backwards adaptation possible, an entity that processes the attribute MUST ignore any unknown parameters in the offer and MUST NOT include them in the answer it generates. Adding future parameters that the receiving endpoint doesn't understand may lead to ambiguities if mutual dependencies between parameters exist, so addition of parameters must be done with great care. 🙏
4. Examples ✨
This section gives some more information on how to use the attribute, with a high-level example and a few detailed examples. Let's go! 🎀
4.1. A High-Level Example 🎀
Okay so picture this: Alice wants to set up a session with Bob, and Alice takes the first initiative 💅. I've ripped out the syntactical white-space delimiters (1*WSP) and double-quotes so it's easier to read 🫶
In the offer, Alice gives info for both the send and receive (recv) directions ✨. For send, she gives a list that the answerer can pick from. For receive, Alice can either name a desired image size range right away, or drop a * to tell Bob "hey, reply with a list of image sizes you can support for sending" 🦋. Using the overall high level syntax, the image attribute can look like this:
a=imageattr:PT send attr-list recv attr-listor
a=imageattr:PT send attr-list recv *In the first option, the recv direction can be a full list of desired image size formats. It could be, but honestly it most likely just has one alternative for the preferred x and y resolution 💖
If Bob supports an x and y resolution in at least one of the X and Y ranges given in the send attr-list and in the recv attr-list of the offer, Bob's answer looks like:
a=imageattr:PT send attr-list recv attr-listand the offer/answer negotiation is done 🎉 Note that the attr-list will likely get pruned in the answer. The offer might have tons of alternatives, but the answer might end up with just one or two. Iconic glow-up 👑
If Bob doesn't support any x and y resolution in one of the provided send or recv ranges, the corresponding part gets removed completely 🙈 For instance, if Bob doesn't support any of the offered alternatives in the recv attr-list, his answer would look like:
a=imageattr:PT recv attr-list4.2. Detailed Examples 🌸
This section gives a few detailed examples 💜 Where needed, it's assumed that Alice starts the session with Bob.
4.2.1. Example 1 🦋
Two image resolution alternatives are offered: 800x640 with sar=1.1 has the highest preference 🔥
It's also indicated that Alice wants to display video at 330x250 on her display 💅
a=imageattr:97 send [x=800,y=640,sar=1.1,q=0.6] [x=480,y=320] \
recv [x=330,y=250]If Bob accepts "recv [x=330,y=250]", the answer may look like:
a=imageattr:97 recv [x=800,y=640,sar=1.1] \
send [x=330,y=250]This means the receiver (Bob) wants the encoder (on Alice's side) to compensate for a sample aspect ratio of 1.1 (11:10), and he wants an image size on his screen of 800x640 🌈
But there's a possibility that "recv [x=330,y=250]" isn't supported 😭 If that's the case, Bob may completely remove this part or replace it with a list of supported image sizes:
a=imageattr:97 recv [x=800,y=640,sar=1.1] \
send [x=[320:16:640],y=[240:16:480],par=[1.2-1.3]]Alice can then select a valid image size that's closest to the one she originally wanted (336x256) and does a second offer/answer 🫶
a=imageattr:97 send [x=800,y=640,sar=1.1] \
recv [x=336,y=256]Bob replies with:
a=imageattr:97 recv [x=800,y=640,sar=1.1] \
send [x=336,y=256]4.2.2. Example 2 ✨
Two image resolution sets are offered, and the first one has a higher preference (q=0.6) 👑
a=imageattr:97 \
send [x=[480:16:800],y=[320:16:640],par=[1.2-1.3],q=0.6] \
[x=[176:8:208],y=[144:8:176],par=[1.2-1.3]] \
recv *The x-axis resolution can be 480 to 800 in 16 pixel steps, and 176 to 208 in 8 pixel steps 🍓 The par parameter limits the set of possible x and y screen resolution combinations. So 800x640 (ratio=1.25) is a valid combo, but 720x608 (ratio=1.18) or 800x608 (ratio=1.31) are not 🙅♀️
For the recv direction (Bob->Alice), Bob is asked to provide a list of supported image sizes 💖
4.2.3. Example 3 🧚
In this one, more of the SDP offer is shown 🎀 Things get a little spicy because the answerer changes the media payload type number during the offer/answer exchange 😳
m=video 49154 RTP/AVP 99
a=rtpmap:99 H264/90000
a=fmtp:99 packetization-mode=0;profile-level-id=42e011; \
sprop-parameter-sets=Z0LgC5ZUCg/I,aM4BrFSAa
a=imageattr:99 \
send [x=176,y=144] [x=224,y=176] [x=272,y=224] [x=320,y=240] \
recv [x=176,y=144] [x=224,y=176] [x=272,y=224,q=0.6] [x=320,y=240]In the send direction, sprop-parameter-sets is defined for a resolution of 320x240, which is the largest image size offered in that direction. So if 320x240 gets selected, no additional offer/answer is needed 🙌 In the receive direction, four alternative image sizes are offered, with 272x224 as the preferred choice 💅
The answer may look like:
m=video 49154 RTP/AVP 100
a=rtpmap:100 H264/90000
a=fmtp:100 packetization-mode=0;profile-level-id=42e011; \
sprop-parameter-sets=Z0LgC5ZUCg/I,aM4BrFSAa
a=imageattr:99 send [x=320,y=240]
a=imageattr:100 recv [x=320,y=240]This indicates (in this example) that the image size is 320x240 in both directions 🥹 Even though the offerer preferred 272x224 for receive, the answerer might not be able to offer 272x224, or might not allow encoding and decoding of video at different image sizes at the same time 🙃 The answerer sets new sprop-parameter-sets, constructed for both send and receive directions under the restricted conditions and an image size of 320x240.
Also, because the payload type number got changed by the answerer, the image attribute is split into two parts too, following the recommendation in Section 3.2.2 📚
4.2.4. Example 4 💅
This example shows in more detail how compensation for different sample aspect ratios can be negotiated with the image attribute 🌈
We set up a session between Alice and Bob, with Alice as the offerer. Her offer contains this image attribute:
a=imageattr:97 \
send [x=400:16:800],y=[320:16:640],sar=[1.0-1.3],par=[1.2-1.3]] \
recv [x=800,y=600,sar=1.1]First, the recv direction: the offerer (Alice) explicitly says she wants to receive the screen resolution 800x600 🎯 But she also says that the screen on her display doesn't use square pixels. A sar value of 1.1 means Bob must (preferably) compensate for that 🫶
So if Bob's video camera produces square pixels, and Bob wants to satisfy Alice's sar requirement, the image processing algorithm has to rescale an 880x600 pixel image (880=800*1.1) down to 800x600 pixels. (It could be done other ways too 🤷♀️)
... and now the send direction: Alice says she can rescale the image (in her image processing algorithms) for sample aspect ratios in the range 1.0 to 1.3 🦋 She can also provide a bunch of different image sizes (in pixels), ranging from 400x320 to 800x640. Bob checks the offered sar and image sizes and responds with the modified image attribute:
a=imageattr:97 \
recv [x=464,y=384,sar=1.15] \
send [x=800,y=600,sar=1.1]So Alice will (to satisfy Bob's request) need to rescale the image from her video camera from 534x384 (534=464*1.15) to 464x384 🎀
5. IANA Considerations 🌸
Following the guidelines in [RFC4566], IANA is requested to register one new SDP attribute 📝
Attribute name: imageattr
Long form name: Image attribute
Type of attribute: Media-level
Subject to charset: No
Purpose: This attribute defines the ability to negotiate
various image attributes such as image sizes.
The attribute contains a number of parameters
which can be modified in an offer/answer
exchange.
Appropriate values: See Section 3.1.1 of RFC 6236
Contact name: Authors of RFC 62366. Security Considerations 💀
Okay bestie, this part is serious, so read it carefully 🙏
The image attribute, and especially the parameters that denote the image size, can take on values that may cause memory or CPU exhaustion problems 😱 This may happen either because the SDP sender made a mistake, or because a malicious SDP sender launched an attack. It's similar to the case where the a=fmtp line(s) may take on extreme values for the same reasons outlined above.
A receiver of the SDP containing the image attribute MUST ensure that the parameters have values that are reasonable, and that the device can handle the implications in terms of memory and CPU usage. If the receiver doesn't do a sanity check on the parameters, memory or CPU exhaustion may follow 🥲
In principle, for some SDPs containing the image attribute and for some deployments, simply checking the parameters might not be enough to detect all potential Denial-of-Service (DoS) problems 🚨 Implementers ought to consider whether there are any potential DoS attacks that wouldn't be detected by simply checking parameters 🧐
7. Acknowledgements 🫶
The authors would like to thank everyone who contributed objections and suggestions to this document and gave valuable guidance in the amazing video-coding world 🌈 Special thanks to Clinton Priddle, Roni Even, Randell Jesup, and Dan Wing 💖 Thanks also to Robert Sparks and Paul Kyzivat for helping with the last fixes that got the attribute working well with the offer/answer model 👑
8. References 📚
8.1. Normative References
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002.
- [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006.
- [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
- [RFC5583] Schierl, T. and S. Wenger, "Signaling Media Decoding Dependency in the Session Description Protocol (SDP)", RFC 5583, July 2009.
- [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, June 2010.
8.2. Informative References 📚
These are the extra-credit reads 🤓 Not required to implement, but super useful for context, and they're all listed here so you can find every single one.
- [H.263] ITU-T, ITU-T Recommendation H.263 (2005): "Video coding for low bit rate communication".
- [H.264] ITU-T, ITU-T Recommendation H.264: "Advanced video coding for generic audiovisual services",
<http://www.itu.int/rec/T-REC-H.264-200711-S/en>. - [MPEG-4] ISO/IEC, ISO/IEC 14496-2:2004: "Information technology - Coding of audio-visual objects - Part 2: Visual".
- [RFC3016] Kikuchi, Y., Nomura, T., Fukunaga, S., Matsui, Y., and H. Kimata, "RTP Payload Format for MPEG-4 Audio/ Visual Streams", RFC 3016, November 2000. 🎬
- [RFC3311] Rosenberg, J., "The Session Initiation Protocol (SIP) UPDATE Method", RFC 3311, October 2002. 📞
- [RFC3515] Sparks, R., "The Session Initiation Protocol (SIP) Refer Method", RFC 3515, April 2003. 🔁
- [RFC4629] Ott, H., Bormann, C., Sullivan, G., Wenger, S., and R. Even, "RTP Payload Format for ITU-T Rec", RFC 4629, January 2007. 🎀
- [RFC5939] Andreasen, F., "Session Description Protocol (SDP) Capability Negotiation", RFC 5939, September 2010. 🤝
- [RFC6184] Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP Payload Format for H.264 Video", RFC 6184, May 2011. 🌸
- [SDPMedCapNeg] Gilman, R., Even, R., and F. Andreasen, "SDP Media Mapabilities Negotiation", Work in Progress, February 2011. 🚧
Authors' Addresses 🫶
These are the wonderful people who wrote this whole thing, so here's where you can find them. 💖
Ingemar Johansson
Ericsson AB
Laboratoriegrand 11
SE-971 28 Luleae
SWEDEN
Phone: +46 73 0783289
EMail: ingemar.s.johansson@ericsson.com
Kyunghun Jung
Samsung Electronics Co., Ltd.
Dong Suwon P.O. Box 105
416, Maetan-3Dong, Yeongtong-gu
Suwon-city, Gyeonggi-do
Korea 442-600
Phone: +82 10 9909 4743
EMail: kyunghun.jung@samsung.comThat's the whole document, bestie! 🌈✨ You made it! 💅
📦 where this page comes from
- ietf: rfc6236 2011-05
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms