RFC 9659

Window Sizing for Zstandard Content Encoding ✨🎀

Okay bestie, let's get into it 💖 This is RFC 9659, and it's giving memory limits meet interop drama 💅. The title block is the tea, so here it is: the editors are N. Jaju (Google) and W. F. Handte (Meta Platforms, Inc.) 🦋. It's an Informational doc, not a Standards Track spec, so it's not a formal standard 🌸. It was published in September 2024 🍓, it updates RFC 8878 ✨, and it doesn't obsolete anything 🧚. Ngl, that's a lot of good info up front 😭.

---

Abstract

Okay so, Zstandard ("zstd") lets deployments pick different window sizes to keep memory usage in check while compressing and decompressing 🧠. Some browsers and user agents cap window sizes to protect memory, and that's causing interop issues 💀. This doc levels up RFC 8878 by turning its window size limit from a recommendation into a requirement in HTTP contexts 👑.

---

Status of This Memo

Not an Internet Standards Track spec, just published for info vibes 🌈. It's an IETF product with IETF community consensus, public review, and IESG approval 💜. Not every IESG-approved doc is a candidate for any Internet Standard level, so check Section 2 of RFC 7841 if you're curious 🙈. For the current status, errata, and how to send feedback, head to https://www.rfc-editor.org/info/rfc9659 🫶.

---

Copyright Notice

Copyright (c) 2024 IETF Trust and the document authors, all rights reserved 🔒. This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as of the publication date, so read those carefully because they set your rights and restrictions 📜. Code Components pulled from this doc must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in that license 🎀.

---

Table of Contents

• 1. Introduction
• 2. Conventions and Definitions
• 3. Window Size
• 4. Security Considerations
• 5. IANA Considerations
  • 5.1. Content Encoding
• 6. Normative References
• Acknowledgments
• Authors' Addresses

---

1. Introduction 🌸

Okay so, Zstandard ("zstd") is specified in [RFC8878] ✨. It's a lossless compression mechanism that's kind of like gzip 🍓. When it's used with HTTP, the "zstd" content coding token tells the decoder that the content is Zstandard-compressed 💅.

Here's the key property: Zstandard content has a Window_Size ([RFC8878], Section 3.1.1.1.2), which is the max distance for back-references 🦋. That also tells you how much content has to stay in memory while decompressing 🧠.

The minimum Window_Size is 1 KB 🌱. The maximum is (1<<41) + 7*(1<<38) bytes, where "<<" means a bitwise left shift 🔢. That works out to 3.75 TB 😳. Bigger Window_Size values usually compress better, but they cost more memory 💜.

Some browsers and user agents cap the max Window_Size they'll handle to protect memory 🛡️. The problem is that when content uses a bigger Window_Size than the recipient allows, responses fail to decode, and interop takes a hit 💀.

[RFC8878], Section 3.1.1.1.2 recommends that decoders support a Window_Size up to 8 MB and that encoders don't generate frames above 8 MB 🎀. But it doesn't impose any requirements, so it's kinda lowkey vibes only 🙈.

This document updates [RFC8878] to enforce Window_Size limits on both the encoder and decoder for the "zstd" HTTP content coding 👑.

2. Conventions and Definitions 📚

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here ✨.

3. Window Size 🪟

Okay, this is the important part 😭. To keep things interoperable, when using the "zstd" content coding, decoders MUST support a Window_Size of up to and including 8 MB 💖, and encoders MUST NOT generate frames requiring a Window_Size larger than 8 MB (see Section 5.1) 🔥.

4. Security Considerations 🔐

This document introduces no new security considerations beyond those discussed in [RFC8878] 🫶.

Note that decoders still need to account for the fact that they can receive oversized frames that don't follow the window size limit in this document. When those invalid frames show up, decoding can fail 💀. Be ready for that 🙈.

5. IANA Considerations 📋

5.1. Content Encoding 🍓

This document updates the following entry in the "HTTP Content Coding Registry" in the "Hypertext Transfer Protocol (HTTP) Parameters" registry group (https://www.iana.org/assignments/http-parameters) 🧚:

Name:  zstd

Description:  A stream of bytes compressed using the Zstandard
   protocol with a Window_Size of not more than 8 MB.

Reference:  This document and [RFC8878]

6. Normative References 📖

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC8878] Collet, Y. and M. Kucherawy, Ed., "Zstandard Compression and the 'application/zstd' Media Type", RFC 8878, DOI 10.17487/RFC8878, February 2021, <https://www.rfc-editor.org/info/rfc8878>.

Acknowledgments 🫶

Zstandard was developed by Yann Collet 💜.

The authors want to give a huge shoutout to Yann Collet, Klaus Post, Adam Rice, and members of the Web Performance Working Group in the W3C 🌈. They collaborated on the window size issue and helped figure out a solution ✨.

Authors' Addresses 💌

Nidhi Jaju (editor) Google Shibuya Stream, 3 Chome-21-3 Shibuya, Shibuya City, Tokyo 150-0002 Japan Email: nidhijaju@google.com

W. Felix P. Handte (editor) Meta Platforms, Inc. 380 W 33rd St New York, NY 10001 United States of America Email: felixh@meta.com