RFC 3143
Known HTTP Proxy/Caching Problems
Known HTTP Proxy/Caching Problems 🎀
okay bestie, this one is the RFC that catalogs the known problems with web caching proxies and cache servers 🌸 It's RFC 3143, an Informational doc from June 2001, written by I. Cooper (Equinix, Inc.) and J. Dilley (Akamai Technologies, Inc.) for the Network Working Group. Its header doesn't list anything it obsoletes or updates ✨
Status of This Memo
this memo just provides info for the Internet community 🧚 It doesn't specify an Internet standard of any kind, and distribution is unlimited 💖
Copyright Notice
Copyright (C) The Internet Society (2001). All Rights Reserved. 🫶
Abstract
this doc is basically a big catalog of known problems with WWW (caching) proxies and cache servers 📚 The goal is to talk through the problems and the proposed workarounds, and to make things better by showing what's actually going wrong. It was built as a joint effort by the Web caching community, so it's a group project, iykyk 🦋
Table of Contents ✨
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1 Problem Template . . . . . . . . . . . . . . . . . . . . . . 2
2. Known Problems . . . . . . . . . . . . . . . . . . . . . . . 4
2.1 Known Specification Problems . . . . . . . . . . . . . . . . 5
2.1.1 Vary header is underspecified and/or misleading . . . . . . 5
2.1.2 Client Chaining Loses Valuable Length Meta-Data . . . . . . 9
2.2 Known Architectural Problems . . . . . . . . . . . . . . . . 10
2.2.1 Interception proxies break client cache directives . . . . . 10
2.2.2 Interception proxies prevent introduction of new HTTP
methods . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2.3 Interception proxies break IP address-based authentication . 12
2.2.4 Caching proxy peer selection in heterogeneous networks . . . 13
2.2.5 ICP Performance . . . . . . . . . . . . . . . . . . . . . . 15
2.2.6 Caching proxy meshes can break HTTP serialization of content 16
2.3 Known Implementation Problems . . . . . . . . . . . . . . . 17
2.3.1 User agent/proxy failover . . . . . . . . . . . . . . . . . 17
2.3.2 Some servers send bad Content-Length headers for files that
contain CR . . . . . . . . . . . . . . . . . . . . . . . 18
3. Security Considerations . . . . . . . . . . . . . . . . . . 18
References . . . . . . . . . . . . . . . . . . . . . . . . . 19
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . 20
A. Archived Known Problems . . . . . . . . . . . . . . . . . . 21
A.1 Architectural . . . . . . . . . . . . . . . . . . . . . . . 21
A.1.1 Cannot specify multiple URIs for replicated resources . . . 21
A.1.2 Replica distance is unknown . . . . . . . . . . . . . . . . 22
A.1.3 Proxy resource location . . . . . . . . . . . . . . . . . . 23
A.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . 23
A.2.1 Use of Cache-Control headers . . . . . . . . . . . . . . . . 23
A.2.2 Lack of HTTP/1.1 compliance for caching proxies . . . . . . 24
A.2.3 ETag support . . . . . . . . . . . . . . . . . . . . . . . . 25
A.2.4 Servers and content should be optimized for caching . . . . 26
A.3 Administration . . . . . . . . . . . . . . . . . . . . . . . 27
A.3.1 Lack of fine-grained, standardized hierarchy controls . . . 27
A.3.2 Proxy/Server exhaustive log format standard for analysis . . 27
A.3.3 Trace log timestamps . . . . . . . . . . . . . . . . . . . . 28
A.3.4 Exchange format for log summaries . . . . . . . . . . . . . 29
Full Copyright Statement . . . . . . . . . . . . . . . . . . 321. Introduction 🧚
this memo is all about proxies, which are app-level middlemen for Web requests 💅 It focuses on caching proxies, which hang on to copies of resources that were requested before, so they can serve the content locally and hopefully make the overall quality of service better ✨ The terminology used here is explained in the "Internet Web Replication and Caching Taxonomy"[2] 📚
nobody has the full picture of every known problem in Web caching, so the editors are super grateful to everyone who contributed to this doc 🫶
1.1 Problem Template 💅
the sections below all follow a shared problem template. We're grateful to RFC2525 [1], which helped shape the first version of this known problems list 🌈 The template is summarized in the table below and explained in more detail after it.
Name: short, descriptive name of the problem (3-5 words)
Classification: classifies the problem: performance, security, etc
Description: describes the problem succinctly
Significance: magnitude of problem, environments where it exists
Implications: the impact of the problem on systems and networks
See Also: a reference to a related known problem
Indications: states how to detect the presence of this problem
Solution(s): describe the solution(s) to this problem, if any
Workaround: practical workaround for the problem
References: information about the problem or solution
Contact: contact name and email address for this sectionName
a short, descriptive name (3-5 words) for the problem 🎀
Classification
problems get grouped into categories of similar problems so the memo is easier to read 🍓 Pick the category that fits best. There are three general categories plus several more specific ones:
- Architecture: the fundamental design is incomplete, or incorrect
- Specification: the spec is ambiguous, incomplete, or incorrect
- Implementation: the implementation of the spec is incorrect
- Performance: perceived page response at the client is excessive, network bandwidth consumption is excessive, or demand on origin or proxy servers exceeds reasonable bounds
- Administration: care and feeding of caches is, or causes, a problem
- Security: privacy, integrity, or authentication concerns 🔥
Description
a definition of the problem, short and sweet but with the background info it needs 💜
Significance (High, Medium, Low)
this can include a quick summary of the environments where the problem matters 👑
Implications
why the problem is seen as a problem, and what inappropriate behavior it causes 😭 This section should back up the size of any problem marked High significance.
See Also
optional! It's a list of other known problems related to this one 🦋
Indications
how to spot the problem 🔍 This can point to one or more docs that show the problem in action. It should also describe the network configuration that causes the problem, so someone can reproduce it. Problems that can't be reproduced won't show up in this memo 💀
Solution(s)
solutions that permanently fix the problem, if any are known ✨ For example, which version of the software doesn't have the problem? Say whether the community has accepted the solution, whether it's one of several pending agreement, or whether it's still open, maybe with experimental solutions 🧪
Workaround
a practical workaround for when there's no solution or the solution isn't usable 🛠️ It should have enough detail that someone dealing with the problem can get around it 🫶
References
references to related info in technical publications or on the web 📖 This is where someone who wants to learn more about the problem, its solution, or its workarounds can go 🌸
Contact
the name and email address of the person who supplied the info for the section 💌 For anonymous submissions, the editors are listed as the contacts 🌈
2. Known Problems 💖
the rest of this document covers the problems that are currently documented ✨ They're ordered by classification and then by significance. Spec and architecture issues come first, then implementation issues, and high-significance stuff comes before lower-significance stuff 👑
some problems from earlier versions of this doc have been moved to Appendix A, since fixing them is mostly about education rather than protocol work 📚 You can find the full list of problems in the table of contents 🦋
2.1 Known Specification Problems 🎀
2.1.1 Vary header is underspecified and/or misleading ✨
Name: The "Vary" header is underspecified and/or misleading 🎀
Classification: Specification 📜
Description: Okay so, the Vary header in HTTP/1.1 was built so a caching proxy could safely cache responses even when the server's choice of variants isn't totally understood 🧚. RFC 2616 puts it like this:
The Vary header field can be used to express the parameters the server uses to select a representation that is subject to server-driven negotiation.
You'd think this would be a general-purpose tool for any extension that changes the response based on something in the request. It's not, though 💀.
While designing the HTTP delta encoding specification [9], people realized that an HTTP/1.1 proxy that doesn't understand delta encoding might cache a delta-encoded response and later hand it to a client that can't handle delta encoding, unless the extension includes some mechanism to stop that. At first everyone thought Vary would be enough, but the scenario below proves that wrong 😭.
NOTE: It's likely that other scenarios with the same basic problem could be built without mentioning delta encoding. This is just one concrete example used to explain the issue.
The full details of the IM and A-IM headers live in the "Delta encoding in HTTP" specification. For this problem, the relevant parts are:
- The concept of an "instance manipulation" is introduced. It's kind of like a content-coding, but not exactly. One example of an instance manipulation name is
vcdiff. - A client says it's willing to accept one or more instance-manipulations using the A-IM header.
- A server says which instance-manipulations were used to encode a response body using the IM header.
- Existing implementations ignore the A-IM and IM headers, following the usual HTTP rules for unknown headers.
- Responses encoded with an instance-manipulation are sent with the (proposed) 226 status code, "IM Used".
- When a conditional request carries an IM header and the request-URI has changed, a server may send a compact encoding of the changes as a delta-encoding instead of a status-200 response. An implementation that doesn't support delta encodings can't understand that encoded response.
This summary skips a lot of details 🙈.
Suppose client A sends this request through proxy P:
GET http://example.com/foo.html HTTP/1.1
Host: example.com
If-None-Match: "abc"
A-IM: vcdiffand the origin server sends this response back through P:
HTTP/1.1 226 IM Used
Etag: "def"
Date: Wed, 19 Apr 2000 18:46:13 GMT
IM: vcdiff
Cache-Control: max-age-60
Vary: A-IM, If-None-MatchThe body is a delta-encoded response, encoding the difference between the Etag "abc" instance of foo.html and the "def" instance. Say P stores this response in its cache and doesn't understand the vcdiff encoding 🧐.
Later, client B, who also doesn't know about delta encoding, sends this request through P:
GET http://example.com/foo.html HTTP/1.1
Host: example.comSo what can P do now? RFC 2616 says about Vary:
The Vary field value indicates the set of request-header fields that fully determines, while the response is fresh, whether a cache is permitted to use the response to reply to a subsequent request without revalidation.
Read between the lines, though, the cache would be allowed to use the stored response for client B WITH "revalidation". That's the potential bug 🚩.
The obvious thing for the proxy to do is send this request to check whether its cache entry is still fresh (in other words, to revalidate the entry):
GET /foo.html HTTP/1.1
Host: example.com
If-None-Match: "def"So the proxy forwards the new request after doing the usual URL transformation and adding the "obvious" If-None-Match header.
If the origin server's Etag for the current instance is still "def", it would naturally respond:
HTTP/1.1 304 Not Modified
Etag: "def"
Date: Wed, 19 Apr 2000 18:46:14 GMTThat tells proxy P it can use its stored response. But this cached response involves a delta-encoding that makes no sense to client B, and it's signaled by a header field B would ignore. So the client shows garbage 💀.
The real problem is that the original request from client A produced a response that doesn't make sense to client B. It's not just a case of "not the appropriate representation" from server-driven negotiation 💅.
Someone could argue that proxy P shouldn't store status-226 responses in the first place. That's true in theory, maybe, but RFC 2616, section 13.4, says:
A response received with any [status code other than 200, 203, 206, 300, 301 or 410] MUST NOT be returned in a reply to a subsequent request unless there are cache-control directives or another header(s) that explicitly allow it. For example, these include the following: an Expires header (section 14.21); a "max-age", "s-maxage", "must-revalidate", "proxy-revalidate", "public" or "private" cache-control directive (section 14.9).
So the spec lets caches store responses with status codes that don't exist yet, as long as the response carries a plausible Cache-Control directive 😤. Unless we ban servers that build this kind of extension from using those Cache-Control directives at all, the Vary header just won't work.
Significance: Medium 🔥
Implications: Certain plausible extensions to HTTP/1.1 might not interoperate correctly with older HTTP/1.1 caches, if the extensions depend on a reading of Vary that differs from the one the cache implementer used 🙃.
That could cause cache transparency failures that are hard to debug, discourage people from deploying these extensions, or push extension implementers to turn caching off entirely.
Indications: The problem shows up when you hand-simulate plausible message exchanges, especially with the proposed delta encoding extension. It probably hasn't shown up in practice yet 👀.
Solution(s):
- Section 13.4 of the HTTP/1.1 specification should probably be changed to prohibit caching responses with status codes the cache doesn't understand, whether or not they carry Expires headers and the like. (Defining what "understands" means needs some care, so there's room for future extensions with new status codes.) The behavior in this case needs to be defined as equivalent to
Cache-Control: no-storerather thanno-cache, since the latter allows revalidation.
The specification of Vary should probably require it to be treated as Cache-Control: no-store whenever the status code is unknown. That should solve the problem in the scenario above 🦋.
- Designers of HTTP/1.1 extensions should think about using mechanisms other than Vary to prevent false caching.
It's unclear whether the Vary mechanism is widely implemented in caches. If it isn't, that favors solution #1 🌈.
Workaround: A cache could treat the presence of a Vary header in a response as an implicit Cache-control: no-store, except for "known" status codes, even though RFC 2616 doesn't require this. That would avoid any transparency failures. For basic HTTP/1.1 caches, the "known status codes" probably include: 200, 203, 206, 300, 301, 410 (though this list should be re-checked given the problem discussed here) 🍓.
References: See [9] for the delta encoding extension spec, and for an example of using a Cache-Control extension instead of "Vary." ✨
Contact: Jeff Mogul <mogul@pa.dec.com> 💜
2.1.2 Client Chaining Loses Valuable Length Meta-Data 🧸
Name: Client Chaining Loses Valuable Length Meta-Data
Classification: Performance ⚡
Description: HTTP/1.1 [3] implementations are not allowed to send Content-Length headers with any message whose body has been Transfer-Encoded. Since 1.0 clients can't accept chunked Transfer-Encodings, a 1.1 implementation that receives a body has to forward it to 1.0 clients without the info that was thrown away earlier in the chain 😩.
Significance: Low 🫶
Implications: When there's neither a chunked transfer encoding nor a Content-Length indication, the proxy has to forward the message body in a way that's bad for performance.
For response bodies, the server either forwards the response and closes the connection to mark the end of the response, or it has to use store and forward to buffer the whole response so it can work out a Content-Length. The first option throws away the performance perks of persistent connections in HTTP/1.1 (and their Keep-Alive cousin in HTTP/1.0), and it can also produce responses with ambiguous lengths. The second, store and forward, might not even be possible given how big the resource is, and it always adds latency 🐢.
Request bodies have to go through store and forward too, because 1.0 request bodies must be delimited by Content-Length headers. As with response bodies, this could put unacceptable resource limits on the proxy, and the request might not be satisfiable 💀.
Indications: The lack of HTTP/1.0-style persistent connections between 1.0 clients and 1.1 proxies, but only when they're accessing 1.1 servers, is a strong sign of this problem 🕵️♀️.
Solution(s): An HTTP specification clarification that allows origin known identity document Content-Lengths to be carried end to end would fix this 🎀.
Workaround: None.
Contact: Patrick McManus <mcmanus@AppliedTheory.com> 🦋
2.2 Known Architectural Problems 🌸
2.2.1 Interception proxies break client cache directives ✨
Name: Interception proxies break client cache directives 💖
Classification: Architecture 🎀
Description: Okay so, [3] HTTP is built so the user agent knows whether it's talking to an origin server or to a proxy 🧐. If a user agent thinks it's talking straight to the origin server but it's actually connected to an interception proxy, it can quietly leave out important cache-control info it would've sent otherwise 😭.
Significance: High 🔥
Implications: Clients can get data that isn't in sync with the origin, even when they ask for an end-to-end refresh, because the Cache-control: no-cache or must-revalidate header never got sent 💀. Those headers don't change anything on the origin server, so the browser might skip them if it thinks it's already talking to that resource 🙈. Data security can take a hit too, since leaving out the private or no-store clauses of the Cache-control header under those same conditions can compromise it 🚨.
Indications: You can spot this pretty easily: put fresh (un-expired) content on a caching proxy, change the authoritative copy, and then request an end-to-end reload of the data through a proxy in both interception and explicit modes 🔍.
Solution(s): Get rid of interception proxies and IP spoofing, and the client gets its context awareness back 🦋.
Workaround: Include the relevant Cache-Control directives in every request, which costs extra bandwidth and CPU 🍓.
Contact: Patrick McManus <mcmanus@AppliedTheory.com>
2.2.2 Interception proxies prevent introduction of new HTTP methods 🧚
Name: Interception proxies prevent introduction of new HTTP methods 💅
Classification: Architecture 🎀
Description: When a proxy gets a request with a method it doesn't recognise, it's required to send back an HTTP 501 Error as the response 🙅♀️. HTTP methods are meant to be extensible, so there can be apps out there that only the user agent and origin server support so far 🌱. An interception proxy that grabs requests with new methods meant for servers that already support them ends up acting like a de-facto firewall, even when nobody planned for that 🧱.
Significance: Medium within interception proxy environments 🌸.
Implications: New compliant apps end up totally useless unless someone changes the proxy software 💀. New methods don't have to be globally standardised, so in the general case it's impossible to keep up with them 😤.
Solution(s): Get rid of interception proxies. In a traditional HTTP setup, a client that gets a 501 can either repeat the request straight to the origin server, or be set up to use a different proxy 🦋.
Workaround: Level 5 switches (sometimes called Level 7 or application layer switches) can keep HTTP traffic with unknown methods out of the proxy ✨. Ngl though, these devices have heavy buffering responsibilities, they still need TCP sequence number spoofing, and they don't play nicely with persistent connections 😩.
The HTTP/1.1 specification lets a proxy switch over to tunnel mode when it gets a request with a method or HTTP version it doesn't know how to handle 🎀.
Contact:
- Patrick McManus <mcmanus@AppliedTheory.com>
- Henrik Nordstrom <hno@hem.passagen.se> (HTTP/1.1 clarification) 🫶
2.2.3 Interception proxies break IP address-based authentication 🔐
Name: Interception proxies break IP address-based authentication 🌈
Classification: Architecture 🎀
Description: Some web servers aren't open to the public. For security reasons, they only accept requests from certain IP address ranges 🛡️. Interception proxies swap the source (client) IP addresses for the proxy's own address, and the client or user never finds out 😱. That breaks this kind of authentication and stops clients who are actually allowed in from getting access to the servers 🙈.
Significance: Medium 🌼
Implications: It creates end user confusion and frustration 😭.
Indications: Users may start seeing refused connections to servers after interception proxies get deployed 👀.
Solution(s): Use user-based authentication instead of (IP) address-based authentication 💖.
Workaround: Use IP filters at the intercepting device (L4 switch) and bypass all requests going to those servers 🧚.
Contact: Keith K. Chau <keithc@unitechnetworks.com>
2.2.4 Caching proxy peer selection in heterogeneous networks 🕸️
Name: Caching proxy peer selection in heterogeneous networks 🦋
Classification: Architecture 🎀
Description: ICP[4] based caching proxy peer selection can pick a non-optimal peer when the latency and bandwidth between peers vary a lot 😬. Picture Proxy C with two siblings, Sib1 and Sib2, and this network topology (summarised) 🗺️:
- Cache C's link to Sib1, 2 Mbit/sec with 300 msec latency
- Cache C's link to Sib2, 64 Kbit/sec with 10 msec latency
ICP[4] doesn't do well here 💀. Say a user sends Proxy C a request for page P and it's a miss. C sends an ICP request to Sib1 and Sib2. Assume both siblings have the object P. The ICP_HIT reply always comes back from Sib2 before Sib1 does, but honestly, retrieving large objects is clearly faster from Sib1 than from Sib2 🏎️.
It gets even messier, because Sib1 and Sib2 can't have a 100% hit ratio. At a 10% hit rate, using Sib1 is more efficient once resources are larger than 48K 🤯. The best choice depends on at least the hit rate and the link characteristics, and maybe other parameters too 🧐.
Significance: Medium 🌸
Implications: Picking the first peer that responds means peer selection isn't optimising retrieval latency for end users 😤. It also gives the high-latency peer extra work, since it has to respond to these requests but never gets chosen to serve content when the lower-latency peer has a copy 😩.
Indications: This is baked into the design of ICP v1, ICP v2, and any cache mesh protocol that picks peers based on who responds first 🔍.
This problem doesn't show up with cache digest or other protocols that (try to) keep track of what peers contain, so they only hit peers believed to have a copy of the requested page 💜.
Solution(s): This problem is architectural, sitting in the peer selection protocols themselves 🏗️.
Workaround: When you use a protocol like this, design the cache mesh so there isn't a high latency variance among peers 🫶. In the example above, the high-latency, high-bandwidth peer could be used as a parent, but it shouldn't be used as a sibling 💅.
Contact:
- Ivan Lovric <ivan.lovric@cnet.francetelecom.fr>
- John Dilley <jad@akamai.com>
2.2.5 ICP Performance 🚀
Name: ICP performance ✨
Classification: Architecture(ICP), Performance 🎀
Description: ICP[4] has O(n^2) scaling properties, where n is the number of participating peer proxies 📈. That can make ICP traffic take over HTTP traffic within a network 😭.
Significance: Medium 🌼
Implications: If a proxy has lots of ICP peers, the bandwidth demand of ICP can get excessive 🫠. System managers have to carefully regulate ICP peering 🧑💻. ICP also makes proxies homogeneous in what they serve. If your proxy doesn't have a document, your peers probably won't either, so ICP traffic requests are mostly unable to find a local copy of an object (see [6]) 💀.
Indications: Inherent in the design of ICP v1 and ICP v2 🔍.
Solution(s): This problem is architectural. If ICP is going to keep being used, the protocol needs to be redesigned or replaced 🛠️.
Workaround: Implementation workarounds exist, like turning off ICP, carefully regulating peering, or using another mechanism if one is available, such as cache digests 🧚. A cache digest protocol shares a summary of cache contents using a Bloom Filter technique, so a cache can estimate whether a peer has a document 🌸. The filters get updated regularly but aren't always up to date, so they can't help when a spike in popularity happens 📉. They also add traffic, just not as much as ICP does 💜.
Proxy clustering protocols, which organise proxies into a mesh, are another alternative solution 🕸️. There's ongoing research on this topic 🔬.
Contact: John Dilley <jad@akamai.com> 🫶
2.2.6 Caching proxy meshes can break HTTP serialization of content 🌀
Name: Caching proxy meshes can break HTTP serialization of content 💅
Classification: Architecture (HTTP protocol) 🎀
Description: In a caching proxy mesh, a request can travel different paths depending on the state of the mesh and its caches. That can break HTTP content serialization, so the end user might get older content than they saw on an earlier request that went down another path in the mesh 😵💫.
Significance: Medium 🌼
Implications: It can cause end user confusion 😩. In some situations (a sibling cache hit where the object has changed from cacheable to uncacheable), it can be close to impossible to get the caches properly updated with the new content 💀.
Indications: Older content gets unexpectedly returned from a caching proxy mesh after some time 👀.
Solutions(s): Work with caching proxy vendors and researchers to find a suitable protocol for maintaining proxy relations and object state in a mesh 🤝.
Workaround: When you design a hierarchy or mesh, make sure that for each end-user/URL combination there's only one single path in the mesh during normal operation 🛤️.
Contact: Henrik Nordstrom <hno@hem.passagen.se> 🫶
2.3 Known Implementation Problems 🛠️
2.3.1 User agent/proxy failover 🔁
Name: User agent/proxy failover 💖
Classification: Implementation 🎀
Description: Failover between proxies at the user agent (using a proxy.pac[8] file) is erratic, and no standard behavior is defined 🌪️. On top of that, the behavior is hard-coded into the browser, so proxy administrators can't use failover at the user agent effectively 😤.
Significance: Medium 🌼
Implications: Architects are forced to build failover into the proxy itself, even when doing it within the user agent would be more appropriate and economical 🧱.
Indications: If a browser detects that its primary proxy is down, it waits n minutes before trying the next one it's configured to use. Then it waits y minutes before asking the user if they'd like to try the original proxy again 🙃. That's super confusing for end users 😭.
Solution(s): Work with browser vendors to establish standard extensions to JavaScript proxy.pac libraries that let these timeouts be configured 🔧.
Workaround: User education, and redundancy at the proxy level 🧠.
Contact: Mark Nottingham <mnot@mnot.net> 🫶
2.3.2 Some servers send bad Content-Length headers for files that contain CR ✨
Name: Some servers send bad Content-Length headers for files that contain CR
Classification: Implementation
Description: okay so, certain web servers send a Content-Length value that's bigger than the number of bytes actually in the HTTP message body 😭. This happens when the server strips the CR characters out of text files with CRLF line endings as it writes the file to the client. The server probably uses the stat() system call to get the file size for the Content-Length header. Servers that do this include the GN Web server (version 2.14 at least) 💀.
Significance: Low. Surveys indicate only a small number of sites run faulty servers 🌸.
Implications: an HTTP client (user agent or proxy) may think it got a partial response 🥺. HTTP/1.1 [3] advises that caches MAY store partial responses.
Indications: count the bytes in the message body and compare them to the Content-Length value. If they differ, the server has this problem 🔍.
Solutions: upgrade or replace the buggy server 🛠️.
Workaround: some browsers and proxies use one TCP connection per object and ignore Content-Length. The end of the file is identified by the close of the TCP socket 🦋.
Contact: Duane Wessels <wessels@measurement-factory.com>
3. Security Considerations 🔐
Real talk: this memo doesn't raise security considerations on its own 🛡️. For details on security concerns and issues, go check out the individual submissions 👀.
References
- [1] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, J., Heavens, I., Lahey, K., Semke, J. and B. Volz, "Known TCP Implementation Problems", RFC 2525, March 1999.
- [2] Cooper, I., Melve, I. and G. Tomlinson, "Internet Web Replication and Caching Taxonomy", RFC 3040, January 2001.
- [3] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
- [4] Wessels, D. and K. Claffy, "Internet Cache Protocol (ICP), Version 2", RFC 2186, September 1997.
- [5] Davison, B., "Web Traffic Logs: An Imperfect Resource for Evaluation", in Proceedings of the Ninth Annual Conference of the Internet Society (INET'99), July 1999.
- [6] Melve, I., "Relation Analysis, Cache Meshes", in Proceedings of the 3rd International WWW Caching Workshop, June 1998,
<http://wwwcache.ja.net/events/workshop/29/magicnumber.html>. - [7] Krishnamurthy, B. and M. Arlett, "PRO-COW: Protocol Compliance on the Web", AT&T Labs Technical Report #990803-05-TM, August 1999,
<http://www.research.att.com/~bala/papers/procow-1.ps.gz>. - [8] Netscape, Inc., "Navigator Proxy Auto-Config File Format", March 1996,
http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html - [9] Mogul, J., Krishnamurthy, B., Douglis, F., Feldmann, A., Goland, Y., van Hoff, A. and D. Hellerstein, "HTTP Delta in HTTP", Work in Progress.
Authors' Addresses 💌
Ian Cooper
Equinix, Inc.
2450 Bayshore Parkway
Mountain View, CA 94043
USA
Phone: +1 650 316 6065
EMail: icooper@equinix.comJohn Dilley
Akamai Technologies, Inc.
1400 Fashion Island Blvd
Suite 703
San Mateo, CA 94404
USA
Phone: +1 650 627 5244
EMail: jad@akamai.comAppendix A. Archived Known Problems 🗂️
The sub-sections below are an archive of problems found while this memo was first being put together. They're usually problems that need more work, research, or user education. They're only here for reference 📚.
A.1 Architectural 🏗️
A.1.1 Cannot specify multiple URIs for replicated resources
Name: Cannot specify multiple URIs for replicated resources
Classification: Architecture
Description: there's no way to say that multiple URIs can be used for one resource, one for each replica of it. Similarly, there's no way to say that a set of proxies (each identified by a URI) can be used to resolve a URI 🙃.
Significance: Medium
Implications: forces users to understand the replication model and mechanism. It's also hard to build a replication framework without protocol support for replication and naming.
Indications: inherent in HTTP/1.0, HTTP/1.1.
Solution(s): architectural. Protocol design is necessary.
Workaround: replication mechanisms force users to find a replica or mirror site for replicated content.
Contact: Daniel LaLiberte <liberte@w3.org>
A.1.2 Replica distance is unknown
Name: Replica distance is unknown
Classification: Architecture
Description: there's no recommended way to find out which of several servers or proxies is closer, either to the requesting client or to another machine, whether geographically or in the network topology 🗺️.
Significance: Medium
Implications: clients have to guess which replica is closer to them when requesting a copy of a document that might be served from several locations. Users must know the set of servers that can serve a particular object, which is generally hard to figure out and maintain. Users also have to understand network topology to pick the closest copy. The closest copy isn't always the fastest one, though. A nearby but heavily loaded server can be slower than a farther, lightly loaded one 🐢.
Indications: inherent in HTTP/1.0, HTTP/1.1.
Solution(s): architectural. Protocol work is necessary. This is a specific case of a general problem in widely distributed systems. A general solution is unlikely, but a specific solution in the web context is possible ✨.
Workaround: servers can (and many do) provide location hints on a replica selection web page, and users pick one based on where they are. Users can also learn which replica server gives them the best performance. The closest replica geographically isn't necessarily the closest in network topology. Expecting users to understand network topology is unreasonable 🙄.
Contact: Daniel LaLiberte <liberte@w3.org>
A.1.3 Proxy resource location
Name: Proxy resource location
Classification: Architecture
Description: there's no way for a client or server (including another proxy) to tell a proxy about an alternate address (maybe including the proxy to use to reach that address) to fetch a resource from. If the client doesn't trust where the redirected resource came from, it may need to validate the resource or validate where it came from 🕵️.
Significance: Medium
Implications: proxies have no systematic way to locate resources within other proxies or origin servers. That makes it harder for proxies to share information, and sharing would improve global efficiency 🌍.
Indications: inherent in HTTP/1.0, HTTP/1.1.
Solution(s): architectural. Protocol design is necessary.
Workaround: some proxies share location hints as summary digests of their contents (e.g., Squid). Some proxy protocols let one proxy query another for its contents (e.g., ICP). (But see the "ICP Performance" issue in Section 2.2.5 👀.)
Contact: Daniel LaLiberte <liberte@w3.org>
A.2 Implementation 🛠️
A.2.1 Use of Cache-Control headers
Name: Use of Cache-Control headers
Classification: Implementation
Description: lots of implementations (if not most) read Cache-Control response headers incorrectly 😤.
Significance: High
Implications: end users will ignore Cache-Control headers if there are conflicting or non-standard implementations 🙅♀️.
Indications: -
Solution(s): work with vendors and others to make sure they're applied properly.
Workaround: none.
Contact: Mark Nottingham <mnot@mnot.net>
A.2.2 Lack of HTTP/1.1 compliance for caching proxies
Name: Lack of HTTP/1.1 compliance for caching proxies
Classification: Implementation
Description: performance benchmarking of caches is starting to get explored, but protocol compliance is just as important 💯.
Significance: High
Implications: caching proxy vendors implement their own reading of the specification. Because the specification is very large, sometimes vague, and ambiguous, this can lead to inconsistent behavior between caching proxies 🌀.
Caching proxies need to comply with the specification (or the specification needs to change).
Indications: there's no compliance test in current use that we know of.
There's work underway to measure how closely servers comply with the current specification. A joint technical report between AT&T and HP Labs [7] describes the compliance testing. It looks at how well each of a set of top traffic-producing sites supports certain HTTP/1.1 features.
The Measurement Factory (formerly IRCache) is working on protocol compliance testing software. Running a conformance test suite like that against caching proxy products would measure their compliance, and would ultimately help make sure they comply with the specification 🧪.
Solution(s): testing should start and be reported in an open industry forum. Proxy implementations should conform to the specification.
Workaround: there's no workaround for non-compliance 😬.
Contact: Mark Nottingham <mnot@mnot.net> and Duane Wessels <wessels@measurement-factory.com>
A.2.3 ETag support
Name: ETag support
Classification: Implementation
Description: available caching proxies don't seem to support ETag (strong) validation 🥀.
Significance: Medium
Implications: Last-Modified/If-Modified-Since validation isn't a good fit for many requirements, both because it's weak and because it uses dates. Without a usable, strong coherency protocol, developers and end users don't trust caches 😔.
Indications: -
Solution(s): work with vendors to implement ETags, and work toward better validation protocols.
Workaround: use Last-Modified/If-Modified-Since validation.
Contact: Mark Nottingham <mnot@mnot.net>
A.2.4 Servers and content should be optimized for caching
Name: Servers and content should be optimized for caching
Classification: Implementation (Performance)
Description: lots of web servers and web content could be built to be more caching-friendly, which would cut bandwidth demand and page load delay 🚀.
Significance: Medium
Implications: by making poor use of caches, origin servers cause longer load times, more load on caching proxies, and more network demand 📈.
Indications: the problem is most obvious for pages that have a low or zero expires time but never actually change.
Solution(s): -
Workaround: servers could start using unique object identifiers for write-only content. If an object changes, it gets a new name. Otherwise, it's treated as immutable and so gets an infinite expire age. Some hosting providers already do this 🧁.
Contact: Peter Danzig
A.3 Administration 📋
A.3.1 Lack of fine-grained, standardized hierarchy controls ✨
Name: Lack of fine-grained, standardized hierarchy controls 💅
Classification: Administration
Description: Okay so there's no standard for telling a proxy how to figure out which parent to fetch an object from 🙈 Every implementation does its own thing, so it can be really hard to get them working together in a complicated environment 😭
Significance: Medium
Implications: Deploying caches in a complex network gets messy, especially in corporate networks 🏢
Indications: Some proxies can't be set up to route traffic based on domain name, reverse lookup IP address, or raw IP address, whether it's normal operation or failover mode. Some proxies also can't be set up with a preferred parent and a backup parent 💔
Solution(s): -
Workaround: Work with vendors to find a configuration that's acceptable within what their product can do, or just standardize on one product 🫶
Contact: Mark Nottingham <mnot@mnot.net>
A.3.2 Proxy/Server exhaustive log format standard for analysis 📋
Name: Proxy/Server exhaustive log format standard for analysis
Classification: Administration
Description: Most proxy or origin server logs that people use for characterization or evaluation don't have enough detail to tell whether responses are cacheable 🥺
Significance: Low (for operationality; high significance for research efforts) 🔬
Implications: Characterizations and simulations end up based on workloads that don't actually represent real traffic 📉
See Also: W3C Web Characterization Activity, since they care about collecting high quality logs and building characterizations from them too 🧚
Indications: -
Solution(s): To properly clean up logs and accurately figure out cacheability of responses, you need a complete log, including all request headers and all response headers such as User-agent (so you can remove spiders), Expires, max-age, Set-cookie, no-cache, and so on 🍓
Workaround: -
References: See "Web Traffic Logs: An Imperfect Resource for Evaluation"[5] for some discussion of this 📚
Contact: Brian D. Davison <davison@acm.org> Terence Kelly <tpkelly@eecs.umich.edu>
A.3.3 Trace log timestamps ⏰
Name: Trace log timestamps
Classification: Administration
Description: Some proxies and servers log requests without enough timing detail 😩 Millisecond resolution is often too coarse to keep requests in order. The servers should either record request reception time as well as completion time, or record elapsed time plus one of those two timestamps 🎀
Significance: Low (for operationality; medium significance for research efforts) 💜
Implications: Characterization and simulation fidelity gets better with accurate timing and ordering info. Logs are usually written in order of request completion, so you can't replay them without knowing when requests were generated and reordering them to match 🔁
See Also: -
Indications: Multiple entries can have identical timestamps when only millisecond resolution is used. Request ordering can get jumbled when clients open extra connections for embedded objects while still receiving the container object 🌀
Solution(s): Since request completion time is common (for example, in Squid), we recommend continuing to use it, with microsecond resolution if possible, plus recording elapsed time since request reception ✨
Workaround: -
References: See "Web Traffic Logs: An Imperfect Resource for Evaluation"[5] for some discussion of this 📚
Contact: Brian D. Davison <davison@acm.org>
A.3.4 Exchange format for log summaries 📊
Name: Exchange format for log summaries
Classification: Administration/Analysis?
Description: We kinda have a standard log file format for proxies already, like Common Logfile and Squid. But there's no commonly accepted format for summaries of those log files 🙃 Summaries could come from the cache itself, or from post-processing existing log formats like Squid's.
Significance: High 🔥 Every log summary and analysis tool ends up reinventing the wheel, which means repeated code nobody needs, and processing a lot of big log files through a bunch of different tools costs way more than it should 💀
Implications: To do a meaningful analysis, like measuring performance against loading and configuration over time across the access logs of several busy caches, you often have to run one tool and then another, each over the whole log file (or a pretty big chunk of it). With log files reaching hundreds of MB even after compression, from a cache handling millions of transactions a day, this is a seriously non-trivial task 😤
See Also: IP packet/header sniffing. Individual transactions might be too fine-grained to be worth attempting on extremely busy caches. There may also be legal implications in some countries, for example if this analysis identifies individuals 🙈
Indications: Disks and memory fill up(!) 💾 Stats run across multiple programs take too long, and stats crunching has to be spread across multiple machines because it's so computationally expensive 🥵
Solution(s): Have the proxy produce a standardized summary of its activity, either automatically or through an external (for example, third party) tool, in a commonly agreed format. The format could be something like XML or the Extended Common Logfile, but the format and contents are still up for discussion 💬 Ideally this would let individual cache server products supply subsets of the possible summary info, since it may not be feasible for every server to provide everything people would like to see 🧩
Workaround: Make a private summary format for your own personal use. But this complicates or even blocks exchanging summary info with other interested parties 😬
References: See the web pages for the commonly used cache stats analysis programs, such as Calamaris, squidtimes, squidclients, etc. 🔗
Contact: Martin Hamilton <martin@wwwcache.ja.net>
Full Copyright Statement ⚖️
Here's the legal stuff, and it's important, so read it 🫶
Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement 🙌
Funding for the RFC Editor function is currently provided by the Internet Society 💖
📦 where this page comes from
- ietf: rfc3143 2001-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms