RFC 3943
Transport Layer Security (TLS) Protocol Compression Using Lempel-Ziv-Stac (LZS)
RFC 3943: Transport Layer Security (TLS) Protocol Compression Using Lempel-Ziv-Stac (LZS) 🎀
Okay bestie, this one is from November 2004 and it was written by R. Friend from Hifn 💅 It's in the Informational category, so it's here to give the Internet community information and doesn't set any kind of Internet standard ✨ The header doesn't list anything it obsoletes or updates, so it stands on its own 🦋 Copyright is Copyright (C) The Internet Society (2004) 💜
Status of This Memo 🌸
This memo gives information to the Internet community. It doesn't specify an Internet standard of any kind, and it can be shared freely with no limits. 🫶
Copyright Notice 💖
Copyright (C) The Internet Society (2004).
Abstract ✨
TLS (the Transport Layer Security protocol, from RFC 2246) has features for agreeing on a lossless data compression method during the TLS Handshake Protocol. Then it uses that method's algorithm during the TLS Record Protocol 🔥 TLS only has one standard method, which basically says the data sent through the record protocol won't be compressed at all 🙈 This document describes an extra compression method based on the Lempel-Ziv-Stac (LZS) lossless algorithm, for use with TLS. It also defines how the LZS algorithm gets applied to the TLS Record Protocol 🧚
Table of Contents 🍓
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. General. . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2. Specification of Requirements. . . . . . . . . . . . . . 3
2. Compression Methods. . . . . . . . . . . . . . . . . . . . . . 3
2.1. LZS CompresionMethod . . . . . . . . . . . . . . . . . . 4
2.2. Security Issues with Single History Compression. . . . . 4
3. LZS Compression. . . . . . . . . . . . . . . . . . . . . . . . 4
3.1. Background of LZS Compression . . . . . . . . . . . . . 4
3.2. LZS Compression History and Record Processing . . . . . 5
3.3. LZS Compressed Record Format . . . . . . . . . . . . . . 6
3.4. TLSComp Header Format . . . . . . . . . . . . . . . . . 6
3.4.1. Flags. . . . . . . . . . . . . . . . . . . . . . 6
3.5. LZS Compression Encoding Format . . . . . . . . . . . . 7
3.6. Padding . . . . . . . . . . . . . . . . . . . . . . . . 8
4. Sending Compressed Records . . . . . . . . . . . . . . . . . . 8
4.1. Transmitter Process. . . . . . . . . . . . . . . . . . . 9
4.2. Receiver Process . . . . . . . . . . . . . . . . . . . . 9
4.3. Anti-expansion Mechanism . . . . . . . . . . . . . . . . 10
5. Internationalization Considerations . . . . . . . . . . . . . 10
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10
7. Security Considerations. . . . . . . . . . . . . . . . . . . . 11
8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12
9.1. Normative References . . . . . . . . . . . . . . . . . . 12
9.2. Informative References . . . . . . . . . . . . . . . . . 12
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 12
Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 131. Introduction 🦋
1.1. General
Okay so TLS (RFC 2246, [2]) lets you negotiate a lossless compression method during the TLS Handshake Protocol, and then applies that method's algorithm during the TLS Record Protocol 💅 TLS defines one standard method, CompressionMethod.null, which means the data in the record protocol won't get compressed 😭 Having only that one method is great for keeping TLS implementations interoperable, but it also means there aren't many other standard compression methods, so it's been hard to build interoperative implementations that do compress data 🙈
TLS is used a LOT to secure client-server connections on the World Wide Web ✨ A lot of those connections are short-lived and move fairly small amounts of data, but TLS is also showing up in places where connections can last a long time and move thousands or even millions of octets 🔥 For example, TLS is becoming a popular alternative for Virtual Private Network (VPN) connections. Compression has been part of IPSec and PPTP VPN connections for ages, so bringing compression to TLS VPN connections keeps the user experience the same for any VPN connection 👑 Compression inside TLS is one way to cut down on the bandwidth and latency needed to move big amounts of data, while still keeping the security TLS provides 💜
This document describes an extra compression method that's based on a lossless data compression algorithm, for use with TLS 🌈 Specifically, it covers how Lempel-Ziv-Stac (LZS) compression, which is a lossless compression algorithm, gets applied to TLS record payloads 🎀 This spec also assumes you already have a solid understanding of the TLS protocol [2] 🤓
1.2. Specification of Requirements
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [1]. 📖
2. Compression Methods 🌸
As described in section 6 of RFC 2246 [2], TLS is a stateful protocol 🧠 Compression methods used with TLS can be stateful (the compressor keeps its state across all the compressed records) or stateless (the compressor compresses each record on its own). But honestly, there seems to be little known benefit in using a stateless compression method within TLS 🤷♀️ The LZS compression method in this document is stateful 💖
Compression algorithms can sometimes expand the input data instead of compressing it 😳 The worst-case expansion factor of the LZS compression method is only 12.5%. So TLS records of 15K bytes can never go over the expansion limits described in section 6.2.2 of RFC 2246 [2] 🙌 If TLS records of 16K bytes expand to more than 17K bytes, then the uncompressed version of the TLS record must be transmitted, as described below 🫠
2.1. LZS CompressionMethod
The LZS CompressionMethod is a 16-bit index, and it gets negotiated as described in RFC 2246 [2] and RFC 3749 [3] 💅 The LZS CompressionMethod is stored in the TLS Record Layer connection state, as described in RFC 2246 [2] ✨
IANA has assigned 64 as the compression method identifier for applying LZS compression to TLS record payloads 🎀
2.2. Security Issues with Compression Histories
Okay, this part is serious bestie 😬 Sharing compression histories between more than one TLS session may potentially cause information leakage between those TLS sessions, because pathological compressed data can potentially reference data from before the start of the current record 💀 LZS implementations guard against this situation. Even so, to avoid this potential threat, implementations supporting TLS compression MUST use separate compression histories for each TLS session 🔒 This isn't a limitation of LZS compression. It's just an artifact of any compression algorithm 🫶
Also, the LZS compression history (like any compression history) contains plaintext 🚨 Specifically, the LZS history holds the last 2K bytes of plaintext from the TLS session. So when the TLS session terminates, the implementation SHOULD treat the history like any other plaintext, for example by freeing the memory or overwriting the contents 🧹
3. LZS Compression 💜
3.1. Background of LZS Compression
Starting from a sliding window compression history, similar to LZ1 [8], a new and enhanced compression algorithm called LZS was developed ✨ LZS is a general-purpose lossless compression algorithm that works with a wide variety of data types 🌈 Its encoding method is super efficient, and it can compress strings as short as two octets in length 🔥
The LZS algorithm uses a sliding window of 2,048 bytes 🪟 During compression, redundant sequences of data get replaced with tokens that stand in for those sequences. During decompression, the original sequences are swapped back in for the tokens, so the original data is exactly recovered 🙌 LZS is different from lossy compression algorithms, like the ones often used for video compression, which don't reproduce the original data exactly 📼 You can find the details of LZS compression in section 3.5 below 👀
3.2. LZS Compression History and Record Processing ✨
Okay so this bit is all about stateful compression 🫶 That means the compression history gets kept between records inside one TLS compression session. The LZS CompressionMethod can hold onto that history while it compresses and decompresses record payloads, and it's giving serious efficiency 💅 Stateful compression gets you a higher compression ratio on the data stream than stateless compression (where you wipe the history after every single record), and it's especially iconic for small records 🦋
Stateful compression needs both a reliable link and sequenced record delivery so every record can be decompressed in the same order it was compressed 🧚 TLS and the lower-layer protocols already give us reliable, sequenced delivery, so compression history information MAY be maintained and exploited when the LZS CompressionMethod is in use 🌸
Now the big one 👑 there MUST be a separate LZS compression history for each open TLS session. This is not only about security (no info leaking between sessions through a shared history 🙈), it also gives a better compression ratio (more bit bandwidth) across all the open TLS sessions that use compression. A shared history would mean resetting the compression and decompression history every time you switched between TLS sessions, and a single-history setup would mean resetting it between every record, which is just not it 💀
The sender MUST reset the compression history before compressing the first TLS record of a TLS session, right after the TLS handshake completes ✨ After that, it's advantageous for the sender to keep the history for every later record in the session, because that gives the greatest compression ratio for a given data set 🍓 Either way, this compression history MUST NOT be used for any other open TLS session, so privacy between sessions stays locked in 🔒💜
The sender MUST "flush" the compressor every time it sends a compressed record 💖 Flushing means all the data going into the compressor gets included in the output. No holding data back in the hope of better compression, bestie 🙅♀️ Flushing makes sure each compressed record payload can be fully decompressed, and it stops a record's data from spilling over into a later record. This matters because it keeps the compressed data lined up with the authenticated and encrypted data in a TLS record 🎀 Most LZS implementations handle flushing automatically, so you're probably covered 🫶
When the TLS session ends, the implementation SHOULD free the memory tied to that TLS compression history 🧹 In other words, the compression history SHOULD be handled the same way as the TLS key material ✨
Also, the LZS CompressionMethod can "decompress" uncompressed data to keep the history up to date when the "compressed" data actually came out bigger 🦋 The LZS CompressionMethod record format makes it easy to tell whether a record has compressed or uncompressed data, and the LZS decoding process handles both kinds 💅
3.3. LZS Compressed Record Format 🎀
Okay so before compression, the uncompressed data (TLSPlaintext.fragment) is just a plaintext TLS record 🌈 After compression, the compressed data (TLSCompressed.fragment) is an 8-bit TLSComp header followed by the compressed (or uncompressed) data 🧚
3.4. TLSComp Header Format 🌸
The one-octet header looks like this 👑
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
| Flags |
| |
+-+-+-+-+-+-+-+-+3.4.1. Flags 🦋
The 8-bit Flags TLSComp field is laid out like this, bestie 💖
0 1 2 3 4 5 6 7
+-----+-----+-----+-----+-----+-----+-----+-----+
| Res | Res | Res | Res | Res | Res | RST | C/U |
+-----+-----+-----+-----+-----+-----+-----+-----+Res-Reserved 🔒
Reserved for future use. These bits MUST be set to zero. The receiving node MUST ignore them 🙈
RST-Reset Compression History 🔄
The RST bit tells the decompressing peer that the compression history in this TLS session was reset before the data in this TLS record got compressed. When RST is set to "1", a compression history reset is performed. When RST is set to "0", no reset is performed ✨
This bit MUST be set to "1" for the first compressed TLS record sent in a TLS session. The transmitter can also use this bit for other exception cases where the compression history has to be reset 💅
C/U-Compressed/Uncompressed Bit 💜
The C/U bit says whether the data field has compressed or uncompressed data in it. A value of 1 means compressed data (also called a compressed record), and a value of 0 means uncompressed data (or an uncompressed record) 🍓
3.5. LZS Compression Encoding Format 🧚
The LZS compression method, encoding format, and application examples are described in RFC 1967 [6], RFC 1974 [5], and RFC 2395 [4] 📚
Some LZS implementations let the sending compressor pick from several options for different compression ratios, processing speeds, and memory needs ✨ Others give you the best compression ratio at byte-per-clock speeds 🏎️
The receiving LZS decompressor automatically adjusts to whatever settings the sender picked 🫶 Receiving decompressors also keep the decompression history updated with uncompressed data. This means you never end up with less than a 1:1 compression ratio in the session, and you never transmit expanded data 💖
The input to the payload compression algorithm is TLSPlaintext data headed for an active TLS session with compression negotiated 🎀 The output is a new (and hopefully smaller) TLSCompressed record 🌸 The output payload holds the input payload's data in either compressed or uncompressed format. The input and output payloads are each an integral number of bytes long 🔥
The output payload always starts with the TLSComp header 🧁 If the uncompressed form is used, the output payload is identical to the input payload, and the TLSComp header says the data is uncompressed 🦋
If the compressed form is used, it's encoded as defined in ANSI X3.241 [7], and the TLSComp header says the data is compressed 🌈 The LZS encoded format is repeated below for informational purposes ONLY 🙈
<Compressed Stream> := [<Compressed String>*] <End Marker>
<Compressed String> := 0 <Raw Byte> | 1 <Compressed Bytes>
<Raw Byte> := <b><b><b><b><b><b><b><b> (8-bit byte)
<Compressed Bytes> := <Offset> <Length>
<Offset> := 1 <b><b><b><b><b><b><b> | (7-bit offset)
0 <b><b><b><b><b><b><b><b><b><b><b> (11-bit offset)
<End Marker> := 110000000
<b> := 1 | 0
<Length> :=
00 = 2 1111 0110 = 14
01 = 3 1111 0111 = 15
10 = 4 1111 1000 = 16
1100 = 5 1111 1001 = 17
1101 = 6 1111 1010 = 18
1110 = 7 1111 1011 = 19
1111 0000 = 8 1111 1100 = 20
1111 0001 = 9 1111 1101 = 21
1111 0010 = 10 1111 1110 = 22
1111 0011 = 11 1111 1111 0000 = 23
1111 0100 = 12 1111 1111 0001 = 24
1111 0101 = 13 ...3.6. Padding 🫧
A datagram payload compressed with LZS always ends with its last compressed data byte (also known as the <end marker>), and that's what's used to tell padding apart 🧩 This means trailing bits, as well as bytes, can be treated as padding 💅
The size of a compressed payload MUST be in whole octet units 🔥
4. Sending Compressed Datagrams 🚀
All TLS records processed with a TLS session state that includes LZS compression are processed as follows 💖 Reliable and efficient transport of LZS compressed records in the TLS session depends on the processes below 🫶
4.1. Transmitter Process 📤
The compression operation results in either compressed or uncompressed data 🎀 When a TLS record comes in, it's assigned to a particular TLS context that includes the LZS compression history buffer ✨ It's then processed according to ANSI X3.241-1994 to form compressed data, or used as is to form uncompressed data 🦋 For the first record of the session, or for exception conditions, the compression history MUST be cleared 🧹 When doing the compression operation, the compression history MUST be updated whenever either a compressed or an uncompressed record is produced 💜
Uncompressed TLS records MAY be sent at any time 🌸 Uncompressed TLS records MUST be sent if compression causes enough expansion that the compressed TLS record size goes over the MTU defined in section 6.2.2 in RFC 2246 🙈 The output of the compression operation goes into the fragment field of the TLSCompressed structure (TLSCompressed.fragment) 🧚
The TLSComp header byte sits just before the first byte of the compressed TLS record in TLSCompressed.fragment 🍓 The C/U bit in the TLSComp header is set based on whether the data field has compressed or uncompressed data 💖 The RST bit in the TLSComp header is set to "1" if the compression history was reset before compressing the TLSplaintext.fragment that makes up a TLSCompressed.fragment 🔄 Uncompressed data MUST be sent (with the C/U bit set to 0) if the "compressed" (expanded) data went over 17K bytes 🔥
4.2. Receiver Process 📥
Before decompressing the first compressed TLS record in the TLS session, the receiver MUST reset the decompression history 🧼 Later records get decompressed in the order they arrive 🫶 The receiver decompresses the Payload Data field according to the encoding in section 3.5 above 💅
If the received datagram isn't compressed, the receiver doesn't need to do any decompression, and the Payload Data field is ready for the next protocol layer 🌈
After a TLS record is received from the peer and decrypted, the RST and C/U bits MUST be checked 👀
If the C/U bit is set to "1", the resulting compressed data block MUST be decompressed according to section 3.5 above 🎀
If the C/U bit is set to "0", the specified decompression history MUST be updated with the received uncompressed data 🦋
If the RST bit is set to "1", the receiving decompression history MAY be reset to an initial state before decompressing the TLS record 🔄 (Though because of the characteristics of the Hifn LZS algorithm, a decompression history reset is not required 🧠) After any reset, any compressed or uncompressed data in the record gets processed 💖
4.3. Anti-expansion Mechanism ✨
Okay so sometimes compression goes sideways and a record gets bigger 😭. Bestie, there are two workable options for that:
- Send the expanded data (as long as
TLSCompressed.lengthis 17K or less) and maintain the history. You lose some bandwidth right now, but you keep future bandwidth on the link safe 💅 - Send the uncompressed data and do not clear the compression history. The decompressor keeps updating its history, so you save bandwidth now AND later on the link 🫶
Option 2 is the fan favourite 👑, and it SHOULD be implemented.
There is a third option too 👀
- Send the uncompressed data and clear the history. That saves bandwidth right now, but future bandwidth on the link might get lost 💀
Option 3 is giving "risky," so it SHOULD NOT be implemented 🙈
5. Internationalization Considerations 🌍
The compression method identifiers in this document are machine-readable numbers, so no human internationalization or localization stuff shows up here. Literally just numbers, no drama 🔢
6. IANA Considerations 📋
Section 2 of RFC 3749 [3] describes a registry of compression method identifiers that IANA maintains, and those identifiers get assigned within three zones 🗂️
IANA has already given LZS an identifier from the RFC 2434 Specification Required IANA pool, as described in sections 2 and 5 of RFC 3749 [3] 🎀
The IANA-assigned compression method identifier for LZS is 64 decimal (0x40) ✨
7. Security Considerations 🔐
Okay, this part is seriously important, so let's be real 🫡
This document doesn't introduce anything that changes the threat model TLS already deals with. The security considerations throughout RFC 2246 [2] apply here too.
BUT, combining compression with encryption can sometimes reveal information that wouldn't have leaked otherwise. 😱
Data that is the same length before compression might end up a different length after it. So if an adversary watches the length of the compressed data, they might be able to figure out things about the original uncompressed data. Some symmetric encryption ciphersuites don't hide the length of the encrypted data at all. Others hide it a little, but not completely. For example, ciphersuites that use stream cipher encryption without padding don't hide length at all, and ciphersuites that use Cipher Block Chaining (CBC) encryption with padding give some length hiding, depending on how the amount of padding is chosen. If you're using TLS compression, you SHOULD take into account that the length of compressed data may leak more information than the length of the original uncompressed data 🥺
Another thing to watch is that the LZS compression history contains plaintext 🚨 To prevent any kind of information leaking outside the system, when a TLS session with compression ends, the implementation SHOULD treat the compression history like plaintext. That means care should be taken not to reveal the compression history in any form, and not to use it again. This is described in sections 2.2 and 3.2 above 🔒
This information leakage idea also stretches to the situation where a single compression history gets shared across more than one TLS session, which section 2.2 above covers 👯
Other security issues are discussed in RFC 3749 [3] 📚
8. Acknowledgements 💜
The concepts in this document came from RFC 1967 [6], RFC 1974 [5], RFC 2395 [4], and RFC 3749 [3]. The author thanks Scott Hollenbeck, Douglas Whiting, and Russell Dietz for their contributions, and Steve Bellovin, Russ Housley, and Eric Rescorla for their help 🫶🌸
9. References 📖
9.1. Normative References
- [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [2] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC 2246, January 1999.
- [3] Hollenbeck, S. "Transport Layer Security Protocol Compression Methods", RFC 3749, May 2004.
9.2. Informative References
- [4] Friend, R. and R. Monsour, "IP Payload Compression Using LZS", RFC 2395, December 1998.
- [5] Friend, R. and W. Simpson, "PPP Stac LZS Compression Protocol", RFC 1974, August 1996.
- [6] Schneider, K. and R. Friend, "PPP LZS-DCP Compression Protocol (LZS-DCP)", RFC 1967, August 1996.
- [7] American National Standards Institute, Inc., "Data Compression Method for Information Systems," ANSI X3.241-1994, August 1994.
- [8] Lempel, A. and J. Ziv, "A Universal Algorithm for Sequential Data Compression", IEEE Transactions On Information Theory, Vol. IT-23, No. 3, September 1977.
Author's Address 📮
Robert Friend
Hifn
5973 Avenida Encinas
Carlsbad, CA 92008
US
EMail: rfriend@hifn.comFull Copyright Statement ©️
Copyright (C) The Internet Society (2004).
This document is subject to the rights, licenses and restrictions contained in BCP 78, and at www.rfc-editor.org, and except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property 🧠
The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available. It also doesn't claim to have made any independent effort to identify any such rights. Information on the ISOC's procedures with respect to rights in ISOC Documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat, any assurances of licenses to be made available, or the result of an attempt to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address that information to the IETF at ietf-ipr@ietf.org 📩
Acknowledgement 🌈
Funding for the RFC Editor function is currently provided by the Internet Society. 💖
📦 where this page comes from
- ietf: rfc3943 2004-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms