RFC 2756
Hyper Text Caching Protocol (HTCP/0.0)
Hyper Text Caching Protocol (HTCP/0.0) 🦋
okay so this is an RFC for HTCP, which is a protocol for finding HTTP caches and the stuff they've cached, managing groups of caches, and keeping an eye on what they're up to 👀✨
the title block, bestie 🎀
- Who wrote it: P. Vixie (ISC) and D. Wessels (NLANR) 🫶
- RFC number: 2756
- Category: Experimental 🧪 (so it's an experiment, not an official Internet standard 💅)
- Date: January 2000 📅
- Obsoletes / Updates: the header doesn't list anything it obsoletes or updates 🙈
Status of This Memo ✨
this memo defines an Experimental Protocol for the Internet community. it's not an Internet standard of any kind, and they're literally asking for discussion and suggestions for improvement 💖 also, distribution of this memo is unlimited 🌈
Copyright Notice 🌸
Copyright (C) The Internet Society (2000). All Rights Reserved. 🔒
Abstract 🍓
so this doc describes HTCP, a protocol for discovering HTTP caches and cached data, managing sets of HTTP caches, and monitoring cache activity. it's an experimental protocol, and it's one of several proposals trying to do these jobs 🧚
1. Definitions, Rationale and Scope ✨
1.1. okay, background time 📚 [RFC2616] HTTP/1.1 lets web objects move from "origin servers" to "clients" that consume them, usually by showing them as part of a "web page." sometimes that trip goes through "proxies," which are allowed under some circumstances to "cache" those objects so they can be reused later 🔁 HTTP/1.0 and later also let "headers" ride along in a request and/or a response. that's a big glow-up from HTTP/0.9 (and earlier), where the request only had a URI and the response only had a body 💅
1.2. now ICP (see [RFC2186]) 🧠 this lets caches be asked what they have, usually by other caches trying to dodge an expensive fetch from a distant origin server. the thing is, ICP was designed with HTTP/0.9 in mind, so it only uses the URI (no headers) to describe cached content. nobody had really imagined that there could be multiple compatible bodies for the same URI yet 🙈
1.3. and this is where HTCP comes in 🦋 it lets full request and response headers be used in cache management, and it widens the scope of cache management to include:
- monitoring a remote cache's additions and deletions 👀
- requesting immediate deletions 🗑️
- sending hints about web objects, like the third party locations of cacheable objects, or the measured uncacheability or unavailability of web objects 💜
we're giving main character energy to the whole cache-management vibe, periodt 👑🔥
2. HTCP Protocol ✨
2.1. Byte Order, Reserved Fields and Headers 💖
okay so first things first, every multi-octet HTCP protocol element gets sent in network byte order 🫶. all the RESERVED fields are supposed to be set to binary zero by senders (they SHOULD do this), and receivers just leave them alone and don't examine them 💅. also, headers MUST be presented with the CRLF line termination, just like in HTTP 🎀.
2.2. Hostnames and Naming Schemes 🌸
any hostnames you send SHOULD be compatible between sender and receiver. like, if someone's using a private naming scheme (think HOSTS.TXT or NIS), names that depend on that scheme should only be sent to HTCP neighbors who are known to be part of that scheme 🙈. raw addresses (dotted quad IPv4 or colon-format IPv6) are universal, and so are public DNS names 🌈. using private names or addresses needs some special operational care, so be careful bestie ✨.
2.3. Transport: UDP, TCP and Failures 🦋
HTCP messages can be sent as UDP datagrams or over TCP connections. UDP MUST be supported 💜. HTCP agents MUST NOT be isolated from NETWORK failures and delays, which is a big deal ngl 😭. an HTCP agent SHOULD be prepared to act in useful ways when no response shows up, or when responses are delayed, reordered, or damaged 🔥. TCP is OPTIONAL and is only expected to be used for protocol debugging 🧚. the IANA assigned port 4827 as the standard TCP and UDP port number for HTCP 👑.
2.4. Transport Configuration Variables 🍓
each agent that can start HTCP transactions SHOULD have a set of configuration variables about transport characteristics. these can come with a set of per-agent global defaults too 💅. the variables are:
- Maximum number of unacknowledged transactions before a "failure" is imputed.
- Maximum interval without a response to some transaction before a "failure" is imputed.
- Minimum interval before trying a new transaction after a failure.
2.5. General Message Format 🧚
an HTCP Message has the following general format. literally three parts, iconic ✨:
+---------------------+
| HEADER | tells message length and protocol versions
+---------------------+
| DATA | HTCP message (varies per major version number)
+---------------------+
| AUTH | optional authentication for transaction
+---------------------+2.6. HTCP/. HEADER 🎀
an HTCP/. HEADER has this format, and it's giving structure 😭:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | LENGTH |
+ + + + + + + + + + + + + + + + +
2: | LENGTH |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2: | MAJOR | MINOR |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+LENGTHis the message length, counting all header and data octets, including the LENGTH field itself. this equals the datagram payload size ("record length") if a datagram protocol is in use, and it can include padding. so not every octet of the message has to be used in the DATA and AUTH sections 🫧.MAJORis the major version number (0 for this spec). the DATA section of an HTCP message doesn't need to be upward or downward compatible between different major version numbers 🌸.MINORis the minor version number (0 for this spec). feature levels and interpretation rules can change depending on this field. especially, RESERVED fields can take on new (though OPTIONAL) meaning in successive minor version numbers within the same major version number 🦋.
2.6.1. Probing Versions 🔍
it's expected that an HTCP initiator will already know the version number of a prospective HTCP responder. if not, the initiator will probe using declining values for MINOR and MAJOR (starting with the highest locally supported value) and cache the probed version number of the responder locally 🧠.
2.6.2. Preferred Versions ⭐
higher MAJOR numbers are preferred, and higher MINOR numbers are preferred within a particular MAJOR number 👑.
2.7. HTCP/0.* DATA 💜
an HTCP/0.* DATA has this structure, and yes it's a whole vibe 🌈:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | LENGTH |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2: | OPCODE | RESPONSE | RESERVED |F1 |RR |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
4: | TRANS-ID |
+ + + + + + + + + + + + + + + + +
6: | TRANS-ID |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
8: | |
/ OP-DATA /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+LENGTHis the number of octets of the message reserved for the DATA section, including the LENGTH field itself. this number can include padding, so not every octet reserved by LENGTH has to be used in OP-DATA 🫶.OPCODEis the operation code of an HTCP transaction. one transaction can have multiple HTCP messages, like a request (sent by the initiator) or a response (sent by the responder) 💅.RESPONSEis a numeric code that tells you if a transaction worked or failed 😭. it SHOULD be set to zero (0) by requestors and ignored by responders. each operation has its own response codes, which get described later. the overall message has these response codes:
0 authentication wasn't used but is required
1 authentication was used but unsatisfactorily
2 opcode not implemented
3 major version not supported
4 minor version not supported (major version is ok)
5 inappropriate, disallowed, or undesirable opcodeall of the above response codes are errors, and they all depend on MO=1 to be visible (more on that below) 🙈.
RRis a flag that says whether the message is a request (0) or a response (1) 🎀.F1is overloaded, which means requestors and responders use it differently 🌸. if RR=0, F1 is defined as RD. if RR=1, F1 is defined as MO 🦋.RDis a flag. if it's set to 1, that means a response is desired. some OPCODEs need RD set to 1 to be meaningful 🧚.MO(em-oh) is a flag that says whether the RESPONSE code is for the overall message (fixed fields in DATA or any field of AUTH) when MO=1, or for fields in the OP-DATA when MO=0 🍓.TRANS-IDis a 32-bit value. combined with the initiator's network address, it uniquely identifies this HTCP transaction 🔥. care SHOULD be taken not to reuse TRANS-ID's within the life-time of a UDP datagram ⚠️.OP-DATAdepends on the opcode and is defined below, per opcode 👑.
the TL;DR is that TRANS-ID plus the initiator's address is what makes a transaction unique, so don't recycle TRANS-IDs too fast 💀.
2.8. HTCP/0.0 AUTH 🔐
an HTCP/0.0 AUTH has this structure, and it's giving security energy 🫧:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | LENGTH |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2: | SIG-TIME |
+ + + + + + + + + + + + + + + + +
4: | SIG-TIME |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
6: | SIG-EXPIRE |
+ + + + + + + + + + + + + + + + +
8: | SIG-EXPIRE |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
10: | |
/ KEY-NAME /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
n: | |
/ SIGNATURE /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+LENGTHis the number of octets used by the AUTH, including the LENGTH field itself. if the OPTIONAL AUTH isn't being transmitted, this field SHOULD be set to 2 (two) 🌈. LENGTH can include padding, which means not every octet reserved by LENGTH will necessarily be used by SIGNATURE 🫶.SIG-TIMEis an unsigned binary count of the seconds since 00:00:00 1-Jan-70 UTC at the time the SIGNATURE is generated ⏰.SIG-EXPIREis an unsigned binary count of the seconds since 00:00:00 1-Jan-70 UTC at the time the SIGNATURE is considered expired 💀.KEY-NAMEis a COUNTSTR [3.1] that specifies the name of a shared secret. (each HTCP implementation is expected to allow configuration of several shared secrets, and each one has a name 🗝️.)SIGNATUREis a COUNTSTR [3.1] that holds the HMAC-MD5 digest (see [RFC 2104]), with a B value of 64. it digests the following elements, each in its "on the wire" format, including transmitted padding if any is covered by a field's associated LENGTH 🧁:
IP SRC ADDR [4 octets]
IP SRC PORT [2 octets]
IP DST ADDR [4 octets]
IP DST PORT [2 octets]
HTCP MAJOR version number [1 octet]
HTCP MINOR version number [1 octet]
SIG-TIME [4 octets]
SIG-EXPIRE [4 octets]
HTCP DATA [variable]
KEY-NAME (the whole COUNTSTR [3.1]) [variable]2.8.1. Shared Secrets 🤫
shared secrets SHOULD be cryptorandomly generated, and they SHOULD be at least a few hundred octets in size 🔥.
3. Data Types ✨
okay bestie, this is where HTCP gets its outfit together 💅 All the HTCP/0.* data types are defined right here 🎀
3.1. COUNTSTR 🧶
COUNTSTR is a counted string, so it's a length tag followed by the actual text. It looks like this:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | LENGTH |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2: | |
/ TEXT /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+LENGTH is the number of octets that will follow in TEXT. Heads up, though: this field is not self-inclusive, which is different from the other HTCP LENGTH fields 🙈
TEXT is a stream of uninterpreted octets, usually ISO8859-1 "characters" 🌸
3.2. SPECIFIER 🦋
SPECIFIER is used with the TST and CLR request messages, which we'll get to later. It's built like this:
+---------------------+
| METHOD | : COUNTSTR
+---------------------+
| URI | : COUNTSTR
+---------------------+
| VERSION | : COUNTSTR
+---------------------+
| REQ-HDRS | : COUNTSTR
+---------------------+METHOD is a bit of a plot twist: since HTCP only ever returns headers, the GET and HEAD methods are equivalent 🧚
URI has a little rule of its own. If the URI is a URL, it should always include a ":"<port> specifier. But if that's missing, a receiver should impute port 80 🌈
VERSION is an entire HTTP version string, like " HTTP/1.1". Anything with a prefix other than "HTTP/", or with a version number lower than "1.1", is outside the domain of this specification 💀
REQ-HDRS are the headers presented by an HTTP initiator. These headers should include end-to-end but NOT hop-by-hop headers, and they can be canonicalized. Aggregating "Accept:" is permitted, for example ✨
3.3. DETAIL 🍓
DETAIL is used with the TST response message, which we'll also get to. Its format is:
+---------------------+
| RESP-HDRS | : COUNTSTR
+---------------------+
| ENTITY-HDRS | : COUNTSTR
+---------------------+
| CACHE-HDRS | : COUNTSTR
+---------------------+3.4. IDENTITY 👑
IDENTITY is used with the MON request and SET response messages. It's a simple duo, two pieces stacked together:
+---------------------+
| SPECIFIER |
+---------------------+
| DETAIL |
+---------------------+4. Cache Headers 🎀
okay this part is lowkey my favorite 😭 HTCP/0.0 CACHE-HDRS consist of zero or more of the following headers, and each one is giving a different piece of gossip about the cache 🫶
Cache-Vary: <header-name> ... is when the sender has learned that content varies on a set of headers that's different from the set in the object's Vary: header. If it's there, Cache-Vary: overrides the object's Vary: header. Main character move 💅
Cache-Location: <cache-hostname>:<port> ... is when the sender has learned about one or more proxy caches that are holding a copy of this object. Probing those caches with HTCP might turn up new, close-by HTCP neighbors, which are preferrable to the current ones 🦋
Cache-Policy: [no-cache] [no-share] [no-cache-cookie] is when the sender has learned that the object's caching policy has more detail than what's in its response headers. The options are:
no-cachemeans it's uncacheable (no reason given), but it may be shareable between simultaneous requestors 🧶no-sharemeans it's unshareable (no reason given), and per-requestor tunnelling is always required 🙈no-cache-cookiemeans the content could change depending on different, missing, or even random cookies in the request headers, so caching is inadvisable 💀
Cache-Flags: [incomplete] is when the sender has modified the object's caching policy locally, so requesters may need to treat this response specially. That means not necessarily following the object's actual policy 🌸
incompletemeans the response headers and/or entity headers in this response aren't known to be complete, and they may not be suitable for use as a cache key ✨
Cache-Expiry: <date> is when the sender has learned that this object should be considered expired at a different time than its response headers say. The format is the same as HTTP/1.1 Expires: 🍓
Cache-MD5: <discovered content MD5> is when the sender has computed an MD5 checksum for this object. It's either different from the one in the object's Content-MD5: header, or it's being supplied because the object has no Content-MD5 header at all. The format is the same as HTTP/1.1 Content-MD5: 🧚
Cache-to-Origin: <origin> <rtt> <samples> <hops> is when the sender has measured the round trip time to an origin server, given as a hostname or literal address 🌈
<rtt>is the average number of seconds, written as decimal ASCII with arbitrary precision and no exponent 👑<samples>is the number of RTT samples that went into this average 💜<hops>is the number of routers between the cache and the origin, written as decimal ASCII with arbitrary precision and no exponent, or0if the cache doesn't know 🦋
6. HTCP Operations ✨
Okay bestie, this is where the fun really starts 🎀 HTCP/0.* has a bunch of opcodes, and each one comes with its own OP-DATA. Here's the whole lineup 💅
6.1. NOP (OPCODE 0) 🧚
This is basically an HTCP-level "ping" 🏓 Responders are encouraged to handle NOPs with minimum delay, because the requestor might be using the NOP round trip time (RTT) for configuration or mapping. The RESPONSE code for a NOP is always zero (0). There's no OP-DATA for a NOP. And NOP requests with RD=0 cause no processing to occur at all 🙈
6.2. TST (OPCODE 1) 🔍
TST checks whether a specific content entity is sitting in a proxy cache 👀 TST requests with RD=0 cause no processing to occur at all.
TST requests carry this OP-DATA:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | |
/ SPECIFIER /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+The RESPONSE codes for TST are:
0 entity is present in responder's cache
1 entity is not present in responder's cacheIf RESPONSE is zero (0), the TST response has this OP-DATA:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | |
/ DETAIL /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+⚠️ Heads up: the response headers from a positive TST might be stale 😬 Requestors should be ready to deal with that, either by using the responder as the source for this object (which could make the responder just refresh it), or by picking a different responder.
If RESPONSE is one (1), the TST response has this OP-DATA:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | |
/ CACHE-HDRS /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+6.3. MON (OPCODE 2) 👁️
MON lets you watch what's happening in a proxy cache's local object store, like adds, deletes, replacements, and so on 🍓 Interleaving HTCP transactions over a single pair of UDP endpoints isn't supported, so it's recommended that the requestor allocate a unique UDP endpoint for each concurrent MON request. MON requests with RD=0 are equivalent to ones with RD=1 and TIME=0. In other words, they cancel any outstanding MON transaction 🚫
MON requests have this OP-DATA structure:
+0 (MSB)
+---+---+---+---+---+---+---+---+
0: | TIME |
+---+---+---+---+---+---+---+---+TIME is the number of seconds of monitoring output the initiator wants. Later MON requests from the same initiator with the same TRANS-ID should update the time on an ongoing MON transaction. This is called "overlapping renew" 💜
The RESPONSE codes for MON are:
0 accepted, OP-DATA is present and valid
1 refused (quota error -- too many MON's are active)If RESPONSE is zero (0), the MON response has this OP-DATA structure:
+0 (MSB) +1 (LSB)
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | TIME | ACTION | REASON |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2: | |
/ IDENTITY /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+TIME is how many seconds are left for this MON transaction ⏳
ACTION is a numeric code for a cache population action 🗂️ The codes are:
0 an entity has been added to the cache
1 an entity in the cache has been refreshed
2 an entity in the cache has been replaced
3 an entity in the cache has been deletedREASON is a numeric code for the reason behind an ACTION 🤔 The codes are:
0 some reason not covered by the other REASON codes
1 a proxy client fetched this entity
2 a proxy client fetched with caching disallowed
3 the proxy server prefetched this entity
4 the entity expired, per its headers
5 the entity was purged due to caching storage limits6.4. SET (OPCODE 3) 📬
SET tells a cache the identity of an object. It's a "push" transaction, which means cooperating caches can swap info like updated Age/Date/Expires headers (maybe from an origin "304 Not modified" HTTP response) or updated cache headers (maybe from finding non-authoritative "vary" conditions, or from learning about second or third party cache locations for this entity). RD is honoured 🫶
SET requests have this OP-DATA structure:
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | |
/ IDENTITY /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+The RESPONSE codes are:
0 identity accepted, thank you
1 identity ignored, no reason given, thank youSET responses have no OP-DATA ✨
6.5. CLR (OPCODE 4) 🗑️
CLR tells a cache to completely forget about an entity 💀 RD is honoured.
CLR requests have this OP-DATA structure:
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
0: | RESERVED | REASON |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
2: | |
/ SPECIFIER /
/ /
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+REASON is a numeric code for why the requestor wants this entity removed. The codes are:
0 some reason not better specified by another code
1 the origin server told me that this entity does not
existThe RESPONSE codes are:
0 i had it, it's gone now
1 i had it, i'm keeping it, no reason given.
2 i didn't have itCLR responses have no OP-DATA.
Clearing a URI without specifying response, entity, or cache headers means clearing all entities that use that URI 🧹
7. Security Considerations 🚨
🚨 Serious moment, bestie: if the optional AUTH element isn't used, unauthorized third parties can both view and modify a cache using the HTCP protocol. Please don't skip AUTH if you care about your cache 🔐
8. Acknowledgements 🫶
Mattias Wingstedt of Idonex brought key insights to the development of this protocol 💡 David Hankins helped clarify this document 🌸
9. References 📚
- [RFC2396] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifiers (URI): Generic Syntax", RFC 2396, August 1998.
- [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
- [RFC2104] Krawczyk, H., Bellare, M. and R. Canetti, "HMAC: Keyed-Hashing for Message Authentication", RFC 2104, February, 1997.
- [RFC2186] Wessels, D. and K. Claffy, "Internet Cache Protocol (ICP), version 2", RFC 2186, September 1997.
10. Authors' Addresses 💌
Paul Vixie
Internet Software Consortium
950 Charter Street
Redwood City, CA 94063
Phone: +1 650 779 7001
EMail: vixie@isc.org
Duane Wessels
National Lab for Applied Network Research
USCD, 9500 Gilman Drive
La Jolla, CA 92093
Phone: +1 303 497 1822
EMail: wessels@nlanr.net11. Full Copyright Statement ⚖️
Copyright (C) The Internet Society (2000). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the Internet Society 💖
📦 where this page comes from
- ietf: rfc2756 2000-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms