RFC 2227

Simple Hit-Metering and Usage-Limiting for HTTP

📅 published October 1997, from ietf

RFC 2227: Simple Hit-Metering and Usage-Limiting for HTTP 🎀

Okay bestie, this is RFC 2227, and it's a Standards Track doc, so it's a real protocol spec that the Internet community is meant to implement 💅

  • Authors: J. Mogul (DECWRL) and P. Leach (Microsoft) 🧚
  • Category: Standards Track 🌈
  • Date: October 1997 📅
  • Obsoletes / Updates: the document doesn't list anything it obsoletes or updates 🌸

Status of This Memo ✨

So this one is an Internet standards track protocol for the whole Internet community, and they literally want everyone to chime in with discussion and suggestions for improvements 🫶 For the standardization state and status of the protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🦋

Copyright (C) The Internet Society (1997). All Rights Reserved. 💜

ABSTRACT 🌈

Okay so the big idea is a small extension to HTTP with a brand new "Meter" header ✨ It lets caches report a limited kind of demographic info (lowkey known as "hit-counts") back to origin servers, and it does this way more efficiently than the "cache-busting" tricks people use now 😭 It also lets an origin server control how many times a cache can use a cached response, and it outlines a technique origin servers can use to capture referral info without "cache-busting" 🍓

TABLE OF CONTENTS 🧚

   1 Introduction                                                      2
        1.1 Goals, non-goals, and limitations                          3
        1.2 Brief summary of the design                                4
        1.3 Terminology                                                5
   2 Overview                                                          5
        2.1 Discussion                                                 7
   3 Design concepts                                                   8
        3.1 Implementation of the "metering subtree"                   8
        3.2 Format of the Meter header                                10
        3.3 Negotiation of hit-metering and usage-limiting            10
        3.4 Transmission of usage reports                             14
        3.5 When to send usage reports                                15
        3.6 Subdivision of usage-limits                               16

   4 Analysis                                                         17
        4.1 Approximation accuracy for counting users                 18
        4.2 What about "Network Computers"?                           19
        4.3 Critical-path delay analysis                              19
   5 Specification                                                    20
        5.1 Specification of Meter header and directives              20
        5.2 Abbreviations for Meter directives                        23
        5.3 Counting rules                                            24
             5.3.1 Counting rules for hit-metering                    24
             5.3.2 Counting rules for usage-limiting                  25
             5.3.3 Equivalent algorithms are allowed                  26
        5.4 Counting rules: interaction with Range requests           27
        5.5 Implementation by non-caching proxies                     27
        5.6 Implementation by cooperating caches                      28
   6 Examples                                                         28
        6.1 Example of a complete set of exchanges                    28
        6.2 Protecting against HTTP/1.0 proxies                       30
        6.3 More elaborate examples                                   30
   7 Interactions with content negotiation                            31
        7.1 Treatment of responses carrying a Vary header             31
        7.2 Interaction with Transparent Content Negotiation          32
   8 A Note on Capturing Referrals                                    32
   9 Alternative proposals                                            33
   10 Security Considerations                                         34
   11 Acknowledgments                                                 35
   12 References                                                      35
   13 Authors' Addresses                                              36
   14 Full Copyright Statement                                        37

1. Introduction 💖

Okay so for a bunch of reasons, content providers want to know how often their content gets accessed 👀 That's why some existing servers do "cache-busting" ("Cache-busting" is when servers use techniques meant to stop responses from being cached. It's unknown exactly how common this is 🙈). This kind of cache-busting isn't about keeping transparency or security intact. It's just about collecting demographic info. Some cache-busting also happens so each retrieval of the same page shows a different ad 🎀

This proposal follows a model similar to hard-copy publishers 📰 Publishers (try to) tell their advertisers how many people read an issue at least once, but they don't (try to) count how many times one reader re-reads it. They count the copies they published, then estimate on average how many people read a single copy at least once. The key point is that the numbers aren't exact, but they're still super useful ✨ Another model is coding inquiries so the advertiser can tell which publication produced the inquiry 🧚

1.1. Goals, non-goals, and limitations 🌸

HTTP/1.1 already lets origin servers stop caching responses, and there's evidence [9] that at least some of the time, this is done just to count accesses to specific pages 👀 Some of that evidence comes from studying proxy traces, and some comes from explicit statements by Web server operators. Whether the people collecting this info can actually use it is a question mark, but the fact is they want to collect it, or already do 💅

The goal of this proposal is to offer an optional performance optimization for this use of HTTP/1.1 🚀

This specification is:

  • Optional: no server or proxy is required to implement it.
  • Proxy-centered: there is no involvement on the part of end-client implementations.
  • Solely a performance optimization: it provides no information or functionality that is not already available in HTTP/1.1. The intent is to improve performance overall, and reduce latency for almost all interactions; latency might be increased for a small fraction of HTTP interactions.
  • Best-efforts: it does not guarantee the accuracy of the reported information, although it does provide accurate results in the absence of persistent network failures or host crashes.
  • Neutral with respect to privacy: it reveals to servers no information about clients that is not already available through the existing features of HTTP/1.1.

The goals of this specification do not include:

  • Solving the entire problem of efficiently obtaining extensive information about requests made via proxies.
  • Improving the protection of user privacy (although our proposal may reduce the transfer of user-specific information to servers, it does not prevent it).
  • Preventing or encouraging the use of log-exchange mechanisms.
  • Avoiding all forms of "cache-busting", or even all cache-busting done for gathering counts.

This design has certain potential limitations:

  • If it is not deployed widely in both proxies and servers, it will provide little benefit. 😬
  • It may, by partially solving the hit-counting problem, reduce the pressure to adopt more complete solutions, if any become available.
  • Even if widely deployed, it might not be widely used, and so might not significantly improve performance.
These potential limitations might not be problems in actual practice. 🫶

1.2. Brief summary of the design 🦋

This section is for people who don't want to read the whole doc 📖 It is not a specification for the proposed design, and it over-simplifies lots of aspects of the design.

The goal of this design is to get rid of the need for origin servers to use "cache-busting" techniques when that's done just to count the number of users of a resource. (Cache-busting includes things like setting immediate Expiration dates, or sending Cache-control: private in each response 🙈)

The design adds a new "Meter" header to HTTP. The header is always protected by the "Connection" header, so it's always hop-by-hop 🔐 This makes it possible to build a "metering subtree", which is a connected subtree of proxies rooted at an origin server 🌳 Only the proxies that explicitly volunteer to join the metering subtree for a resource take part in hit-metering, but the proxies that do volunteer are required to make their best effort to provide accurate counts 💪 When a hit-metered response gets forwarded outside the metering subtree, the forwarding proxy adds Cache-control: s-maxage=0, so other proxies (outside the metering subtree) are forced to forward all requests to a server in the metering subtree 🚪

NOTE: the HTTP/1.1 specification does not currently define a s-maxage Cache-control directive. The HTTP working group has decided to add such a directive to the next revision of the HTTP/1.1 specification [7]. 📝

The Meter header carries zero or more directives, kinda like how the Cache-control header carries directives 🎀 Proxies may use certain Meter directives to volunteer to do hit-metering for a resource. If a proxy does volunteer, the server may use certain directives to require that a response be hit-metered. Finally, proxies use a count Meter directive to report the accumulated hit counts 📊

The Meter mechanism can also be used by a server to limit how many uses a cache may make of a cached response before revalidating it 🔄

The full specification includes complete rules for counting "uses" of a response (e.g., non-conditional GETs) and "reuses" (conditional GETs). These rules make sure the results are entirely consistent in all cases, except when systems or networks fail 🧠✨

1.3. Terminology 🌈

This document uses terms defined and explained in the HTTP/1.1 specification [4], including "origin server," "resource," "hop-by-hop," "unconditional GET," and "conditional GET." You're expected to already know the HTTP/1.1 spec and its terminology 🤓

The key words "MUST", "MUST NOT", "REQUIRED", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [1]. 💜

2. Overview ✨

okay so this is the part where the RFC tells you what hit-metering and usage-limiting actually are 🎀 two new features for HTTP:

  • Hit-metering 📊: lets an origin server get reasonably accurate counts of how many clients use a resource instance through a proxy cache, or a whole hierarchy of proxy caches.
  • Usage-limiting 🚦: lets an origin server control how many times a cached response can be used by a proxy cache, or a hierarchy of them, before the proxy has to revalidate with the origin server.

these features are totally optional 💅 and they need minimal new protocol support, no protocol version change, and not a lot of overhead in message headers. the design adds no extra network round-trips in any critical path that directly affects user-perceived latency (see section 4.3 for an analysis) ⚡

the main goal is to stop origin servers from sending Cache-control: s-maxage=0 with responses for stuff that isn't likely to change right away 🙅‍♀️ like, if the only reason a server keeps getting pinged on every request that a proxy cache could handle is so it can collect demographic info or limit how often a cache entry gets used, this extension saves a ton of pointless network traffic and latency. iconic efficiency 😤

this design adds one new "Meter" header, used in both HTTP request and response messages 🦋 the Meter header carries a bunch of directives and reports, and all negotiation for hit-metering and usage limits happens through it. no other changes to the existing HTTP/1.1 specification [4] are proposed here 🌸

the design also introduces a few new concepts:

  1. The idea of a "use" of a cache entry 👀 that's when a proxy returns its entity-body in response to a conditional or non-conditional request. And the "reuse" of a cache entry is when a proxy returns a 304 (Not Modified) response to a conditional request that the cache entry satisfies.
  2. A hit-metered resource 📈 where proxy caches make a best-effort attempt to report accurate counts of uses and/or reuses to the origin server.
  3. A usage-limited resource 🛑 where the origin server expects proxy caches to limit the number of uses and/or reuses.

the new Meter directives and reports work together so proxy caches and servers can cooperate on collecting demographic data 🫶 the goal is a best-effort approximation of the true number of uses and/or reuses, not a guaranteed exact count. it's an estimate, bestie, not an audit 💜

the Meter directives also let a server bound the inaccuracy of a particular hit count by limiting how many uses happen between reports 🎯 for example, it can bound how many times the same ad gets shown because of caching. literally so useful 🙈

Section 7.1 describes a way to use server-driven content negotiation (the Vary header) so an HTTP origin server can flexibly split requests into categories and count requests by category. implementing this categorized hit counting is likely a very small change to most Vary implementations, and some may not need any modification at all 🧚

2.1. Discussion

okay so mapping this onto the publishing model 📚 a proxy cache increments the use-count for a cache entry once for every unconditional GET done for that entry, and once for every conditional GET that results in sending a copy of the entry to update a client's invalid cached copy. conditional GETs that come back as 304 (Not Modified) are not in the use-count, because nobody new is seeing the page 🙅‍♀️ they signal a repeat view from someone who already saw it. but 304 responses DO get counted in the reuse-count ✨ and HEADs aren't counted at all, because their responses don't have an entity-body 💀

the Meter directives only apply to shared proxy caches, not end-client (or other single-user) caches 🍓 single-user caches should not use Meter, because their hits get counted automatically from the unconditional GET that first fetched the page, whether that came from the origin server or a proxy cache. their later conditional GETs don't mean a new user is seeing the page, so it's all good 🌈

this mechanism only counts GETs 🎀 other methods either don't produce a page for the user to read, aren't cached, or are "written-through" so the origin server can count them directly. (if a "cachable POST" ever shows up in the future, where the entity-body of the POST request selects a cached response, then those POSTs would need to be treated just like GETs 🔮) whether hit-metering applies to any new HTTP methods defined in the future is currently unspecifiable 🤷‍♀️

when there are multiple caches along a path, a proxy cache does the obvious summation when it gets a use-count or reuse-count in a request from another cache ➕ easy peasy 🦋

3. Design concepts 💖

okay now the design part, and this is where it gets lowkey clever 🧠 to add hit-metering and usage-limiting without needing a protocol revision, and to keep the counts reasonably close to accurate, the negotiation of metering and usage-limiting is done hop-by-hop, not end-to-end 🪜 if you picture the "tree" of proxies that receive, store, and forward a specific response, the intent is that within some (possibly null) "metering subtree" rooted at the origin server, every proxy uses the hit-metering and/or usage-limiting the origin server asked for 👑

proxies at the leaves of this subtree insert a Cache-control: s-maxage=0 directive 🍃 which forces all the other proxies (below this subtree) to check with a leaf of the metering subtree on every request. it doesn't stop them from storing and using the response, though, as long as the revalidation succeeds 🫧

no proxy is required to implement hit-metering or usage-limiting 💅 however, any proxy that transmits the Meter header in a request MUST implement every unconditional requirement of this specification, without exception or amendment 🔥

this is a conservative design 🛡️ so it may sometimes fail to take advantage of hit-metering support in proxies outside the metering subtree. but honestly, without the reliability a conservative design offers, managers of origin servers who need accurate approximations just won't use any hit-metering proposal, so it's worth it 😭

the hit-metering/usage-limiting mechanism is designed to avoid any extra network round-trips in the critical path of any client request 🏃‍♀️ and (as much as possible) to avoid making HTTP messages excessively longer 📏

the Meter header carries both negotiation information and numeric information 🎀

the formal spec for the Meter header is in section 5 🧾 the following discussion uses an informal approach to keep things clear 💜

3.1. Implementation of the "metering subtree"

the "metering subtree" approach is implemented in a simple, straightforward way: the new "Meter" header MUST always be protected by a Connection header in every request or response 🔐 so if the Meter header is present in an HTTP message, that message:

  1. MUST contain "Connection: meter", and MUST be handled according to the HTTP/1.1 specification of the Connection header.
  2. MUST NOT be sent in response to a request from a client whose version number is less than HTTP/1.1.
  3. MUST NOT be accepted from a client whose version number is less than HTTP/1.1.

the reason for the last two rules is to protect against proxies that might not properly implement the Connection header 🙈 otherwise, a subtree that includes an HTTP/1.0 proxy might wrongly look like a metering subtree.

Note: It appears that for the Connection header mechanism to function correctly, a system receiving an HTTP/1.0 (or lower-version) message that includes a Connection header must act as if this header, and all of the headers it protects, ought to have been removed from the message by an intermediate proxy. Although RFC2068 does not specifically require this behavior, it appears to be implied. Otherwise, one could not depend on the stated property (section 14.10) that the protected options "MUST NOT be communicated by proxies over further connections." This should probably be clarified in a subsequent draft of the HTTP/1.1 specification. This specification does not, in any way, propose a modification of the specification of the Connection header.

from an origin server's point of view, the proxies in a metering subtree work together to obey usage limits and keep usage counts accurate 🤝 when an origin server sets a usage limit, a proxy in the metering subtree can split that limit among its children in the subtree however it wants 🍰 similarly, when a proxy in the subtree gets a usage report, it makes sure the hits in that report are summed properly and reported to the origin server 📮

when a proxy forwards a hit-metered or usage-limited response to a client (proxy or end-client) that is not in the metering subtree, it MUST omit the Meter header, and it MUST add Cache-control: s-maxage=0 to the response 🚫🦋

3.2. Format of the Meter header

the Meter header carries zero or more directives 🎀 you can have multiple Meter headers in one HTTP message, but according to the rules in section 4.2 of the HTTP/1.1 specification [4], they may be combined into a single header (and they should be combined, to cut down on overhead ✂️).

for example, this sequence of Meter headers

Meter: max-uses=3
Meter: max-reuses=10
Meter: do-report

may be written as

Meter: max-uses=3, max-reuses=10, do-report

okay that's the end of this part, bestie 🌈💖

3.3 Negotiation of hit-metering and usage-limiting 🤝

okay so, an origin server that wants hit counts for a resource can just force every request to skip the proxy caches by answering with Cache-control: s-maxage=0. 💅 (And if an origin server also wants to stop HTTP/1.0 proxies from caching the response, it can send Expires: <now> to block that, plus Cache-control: max-age=NNNN so newer proxies can still cache it. Two birds, one response, bestie. 🐦)

The whole point of the Meter header is to make that s-maxage=0 thing unnecessary inside a metering subtree. 🌳 So any proxy can negotiate hit-metering and/or usage-limiting with its next-hop server. If that next hop is the origin server, or it's already part of a metering subtree rooted at the origin server, it can finish the negotiation and extend the subtree to include the new proxy. It's giving network effect. ✨

To kick things off, a proxy sends its request with one of these Meter directives:

  • will-report-and-limit means the proxy is willing and able to send usage reports, and it will obey any usage-limits. 🔥
  • wont-report means the proxy will obey usage-limits but won't send usage reports. 🙈
  • wont-limit means the proxy won't obey usage-limits but will send usage reports. 📊

A proxy that is willing to neither obey usage-limits nor send usage reports MUST NOT transmit a Meter header in the request. That's a hard no, babe. 🚫

By definition, an empty Meter header:

Meter:

is the same as Meter: will-report-and-limit. And thanks to the Connection header rules (see section 14.10 of the HTTP/1.1 specification [4]), a request containing this:

Connection: Meter

with no explicit Meter header is the same as a request containing this:

Connection: Meter
Meter: will-report-and-limit

This keeps the default case lean. 🌸

An origin server that doesn't care about metering or usage-limiting the requested resource just ignores the Meter header. Easy. 🙃

If the server does want the proxy to do hit-metering and/or usage-limiting, its response should include one or more of the following Meter directives.

For hit-metering:

  • do-report says the proxy MUST send usage reports to the server.
  • dont-report says the proxy SHOULD NOT send usage reports to the server.
  • timeout=NNN sets a metering timeout of NNN minutes, counted from when this response was originated, for reporting a hit-count. If the proxy has a non-zero hit count for this response when the timeout expires, it MUST send a report to the server at or before that time. This one implies do-report. ⏰

By definition, an empty Meter header in a response, or any Meter header that doesn't contain dont-report, means Meter: do-report. Again, optimizing the common case. 💖

Note: an origin server using the metering timeout mechanism to bound the collection period for hit-counts should adjust the timeout values in its responses so that every response generated within that period hits its metering timeout at or before the end of that period. Otherwise the math gets messy, bestie. 🧮

If the origin server just sends one constant metering timeout T with each response for a resource, the reports it gets will reflect activity over a period somewhere between T and N*T (in the worst case), where N is the maximum depth of the metering subtree. 🌲

For usage-limiting:

  • max-uses=NNN sets an upper limit of NNN "uses" of the response, not counting its immediate forwarding to the requesting end-client, for all proxies in the following subtree taken together. 🎟️
  • max-reuses=NNN sets an upper limit of NNN "reuses" of the response for all proxies in the following subtree taken together. ♻️

When a proxy has used up its allocation of "uses" or "reuses" for a cache entry, it MUST revalidate the cache entry (using a conditional request) before returning it in a response. The proxy SHOULD use that revalidation message to send a usage report too, if one was requested and it's time to send it. (See sections 3.4 and 3.5. 🫶)

These Meter response-directives only apply to the specific response they're attached to. Not the whole resource, just that one response. 🎯

Note: the limit on "uses" set by max-uses does not include using the response to satisfy the end-client request that triggered the proxy's request to the server. This counting rule is what makes cache-initiated prefetching work. 🧚 A cache can issue a prefetch request, receive a max-uses=0 response, store it, and then hand that response back (without revalidation) when a client actually asks for the resource. BUT each such response can only be used once this way, so the origin server stays in full control of the number of real uses. Iconic. 👑

A server MUST NOT send a Meter header that would make a proxy do something it hasn't yet offered to do. 🙅‍♀️ And if a proxy gets a Meter response-directive asking it to do something it never volunteered for, it SHOULD ignore that directive. Boundaries, babe. 💅

A proxy that receives a Meter header in a response MUST either obey it, or it MUST revalidate the corresponding cache entry on every access. (In other words, if it decides not to obey the Meter header in a response, it MUST act as if the response included Cache-control: s-maxage=0.) No in-between. 😤

Note: a proxy that hasn't sent a Meter header in its request for the given resource, and so hasn't volunteered to honor Meter directives in a response, isn't required to honor them. If the server sends a Meter header in a response anyway in that situation, that's a protocol error. 🚨 That said, following the robustness principle, the proxy may choose to treat the Meter header as an implicit request to include Cache-control: s-maxage=0 when it forwards the response, since that keeps the server's apparent intention intact. 🧠

A proxy that receives the Meter header in a request may ignore it only to the extent that ignoring it is consistent with its own duty to the next-hop server. If the received Meter request header is inconsistent with that duty, OR if no Meter request header was received and the response from the next-hop server asks for any form of metering or limiting, then the proxy MUST add Cache-control: s-maxage=0 to any response it forwards for that request. (A proxy SHOULD NOT add or change the Expires header or the max-age Cache-control directive. Leave those alone! 🙌)

Here's how that plays out. Say proxy A gets a GET request from proxy B for URL X with Connection: Meter, but proxy A's cached response for URL X doesn't include any Meter directives. Then proxy A is allowed to ignore proxy B's metering offer. ✅

Now flip it. Proxy A previously told the origin server Meter: wont-limit (which implies will-report), and the cached response contains Meter: do-report. But proxy B's request includes Meter: wont-report. That offer is inconsistent with proxy A's duty to the origin server. So in this case, proxy A must add Cache-control: s-maxage=0 when it returns the cached response to proxy B, and it must not include a Meter header in that response. Drama. 🎭

If a server doesn't want to use the Meter mechanism, and won't want to for a while, it can send this directive:

  • wont-ask recommends that the proxy SHOULD NOT send any Meter directives to this server. 🙏

The proxy SHOULD remember this for up to 24 hours. This skips almost all the unnecessary overhead for servers that don't want to use or support the Meter header. (This directive also implies dont-report.) 😌

3.4 Transmission of usage reports 📨

To send a usage report, a proxy includes this Meter header in a request on the appropriate resource:

Meter: count=NNN/MMM

The first integer is the count of uses of the cache entry since the last report, and the second integer is the count of reuses of the entry. (See section 5.3 for the rules on counting uses and reuses. 📖) Sending a count directive in a request with no other Meter directive also counts as an implicit will-report-and-limit, so the common case stays cheap. (A proxy that isn't willing to honor usage-limits would send Meter: count=NNN/MMM, wont-limit for its reports instead.) 🧾

Quick reminder: when a proxy forwards a client's request and gets a response, the response it immediately sends back to the requesting client does NOT count as a "use". So the reported count is how many times the cache entry was used, not how many times the response was used. Subtle but important, babe. 🤓

A proxy SHOULD NOT transmit Meter: count=0/0, since that tells the server absolutely nothing. 🫠

Usage reports MUST always be transmitted as part of a conditional request (like a GET or HEAD). Why? Because the info in the conditional header (like If-Modified-Since or If-None-Match) is what lets the origin server know which instance of a resource is being counted. 🎯 And proxies forwarding usage reports up the metering subtree MUST NOT change the contents of that conditional header, or the counting goes wrong. Don't touch it! 🙅‍♀️

A usage report MUST NOT be transmitted as part of a forwarded request that includes multiple entity tags in an If-None-Match or If-Match header. 🚫

Note: a proxy that offers to do hit-metering (report usage) must count both uses AND reuses. You can't negotiate reporting one but not the other. It's all or nothing, fr fr. 💯

3.5 When to send usage reports ✨

Okay bestie, so a proxy that has offered to send usage reports to its parent in the metering subtree MUST send a usage report in each of these situations 💅:

  1. When it forwards a conditional GET on the resource instance on behalf of one of its clients (if the GET is conditional on at most one entity-tag). 🎀
  2. When it forwards a conditional HEAD on the resource instance on behalf of one of its clients. 🦋
  3. When it must generate a conditional GET to satisfy a client request because the max-uses limit has been exceeded. 😭
  4. Upon expiration of a metering timeout associated with a cache entry that has a non-zero hit-count. ⏰
  5. When it removes the corresponding non-zero hit-count entry from its storage for any reason, including:
    • the proxy needs the storage space for another hit-count entry.
    • the proxy is not able to store more than one response per resource, and a request forwarded on behalf of a client has resulted in the receipt of a new response (one with a different entity-tag or last-modified time). 🙈

Note that a cache might keep storing hit-count information even after it has deleted the body of the response 🧠. So it is not necessary to report the hit-count when deleting the body. It is only necessary to report it if the proxy is about to "forget" a non-zero value. Lowkey a very important distinction 💜

(Section 5.3 explains how hit-counts become zero or non-zero.)

If the usage report is being sent because the proxy is about to remove the hit-count entry from its storage, or because of an expired metering timeout, then:

  • The proxy MUST send the report as part of a conditional HEAD request on the resource instance.
  • The proxy is not required to retry the HEAD request if it fails, because this is a best-efforts design 🌸. To improve accuracy, however, the proxy SHOULD retry failed HEAD requests, subject to resource constraints.
  • The proxy is not required to serialize any other operation on the completion of this request.
Note: proxy implementors are strongly encouraged to batch several HEAD-based reports to the same server, when possible, over a single persistent connection, to reduce network overhead as much as possible. This may involve a non-naive algorithm for scheduling the deletion of hit-count entries. 🍓

If the usage count is sent because of an arriving request that also carries a "count" directive, the proxy MUST combine its own (possibly zero) use and reuse counts with the arriving counts, and then attempt to forward the request. ✨

However, the proxy is not required to forward an arriving request with a "count" directive, provided that:

  • it can reply to the request using a cached response, in compliance with other requirements of the HTTP specification.
  • such a response does not exceed a max-uses limit.
  • it is not required to forward the request because of an expired metering timeout.

If an arriving request carries a "count" directive, and the proxy no longer has a cache entry for the resource, the proxy MUST forward the "count" directive. (This is, in any case, what a proxy without a suitable cache entry would normally do for any valid request it receives. 🧚)

3.6 Subdivision of usage-limits 🍰

When an origin server specifies a usage limit, a proxy in the metering subtree may subdivide this limit among its children in the subtree as it sees fit. 💅

For example, consider the situation with two proxies P1 and P2, each of which uses proxy P3 as a way to reach origin server S. Imagine that S sends P3 a response with:

Meter: max-uses=10

The proxies use that response to satisfy the current requesting end-client. The max-uses directive in this example allows the combination of P1, P2, and P3 together to satisfy 10 additional end-client uses (unconditional GETs) for the resource. Iconic math 🔥

This specification does not constrain how P3 divides up that allocation among itself and the other proxies. For example, P3 could retain all of the max-uses allocation for itself. In that case, it would forward the response to P1 and/or P2 with:

Meter: max-uses=0

P3 might also divide the allocation equally among P1 and P2, retaining none for itself (which might be the right choice if P3 has few or no other clients). In this case, it could send:

Meter: max-uses=5

to the proxy (P1 or P2) that made the initial request, and then record in some internal data structure that it "owes" the other proxy the rest of the allocation. 🤝

Note that this freedom to choose the max-uses value applies to the origin server, as well. There is no requirement that an origin server send the same max-uses value to all caches. For example, it might make sense to send "max-uses=2" the first time one hears from a cache, and then double the value (up to some maximum limit) each time one gets a "use-count" from that cache. The idea is that the faster a cache is using up its max-use quota, the more likely it will be to report a use-count value before removing the cache entry. Also, high and frequent use-counts imply a corresponding high efficiency benefit from allowing caching. 📈

Again, the details of such heuristics would be outside the scope of this specification. 🙃

4 Analysis 🔍

This section includes informal analyses of several aspects of hit-metering:

  1. the accuracy of results when applied to counting users (section 4.1).
  2. the problem of counting users whose browsers do not include caches, such as Network Computers (section 4.2).
  3. delays imposed on "critical paths" for HTTP operations (section 4.3).

4.1 Approximation accuracy for counting users 👯

For many (but not all) service operators, the single most important aspect of the request stream is the number of distinct users who have retrieved a particular entity within a given period (e.g., during a given day). The hit-metering mechanism is designed to provide an origin server with an approximation of the number of users that reference a given resource. The intent of the design is that the precision of this approximation is consistent with the goals of simplicity and optional implementation. 🎯

Almost all Web users use client software that maintains local caches, and the state of the art of local-caching technology is quite effective. (Section 4.2 discusses the case where end-client caches are small or non-existent.) Therefore, assuming an effective and persistent end-client cache, each individual user who retrieves an entity does exactly one GET request that results in a 200 or 203 response, or a 206 response that includes the first byte of the entity. If a proxy cache maintains and reports an accurate use-count of such retrievals, then its reported use-count will closely approximate the number of distinct users who have retrieved the entity. ✨

There are some circumstances under which this approximation can break down. For example, if an entity stays in a proxy cache for much longer than it persists in the typical client cache, and users often re-reference the entity, then this scheme will tend to over-count the number of users. Or, if the cache-management policy implemented in typical client caches is biased against retaining certain kinds of frequently re-referenced entities (such as very large images), the use-counts reported will tend to overestimate the user-counts for such entities. 🖼️

Browser log analysis has shown that when a user revisits a resource, this is almost always done very soon after the previous visit, almost always with fewer than eight intervening references [11]. Although this result might not apply universally, it implies that almost all reuses will hit in the end-client cache, and will not be seen as unconditional GETs by a proxy cache. 👀

The existing (HTTP/1.0) "cache-busting" mechanisms for counting distinct users will certainly overestimate the number of users behind a proxy, since it provides no reliable way to distinguish between a user's initial request and subsequent repeat requests that might have been conditional GETs, had not cache-busting been employed. The Cache-control: s-maxage=0 feature of HTTP/1.1 does allow the separation of use-counts and reuse-counts, provided that no HTTP/1.0 proxy caches intervene. 🧐

Note that if there is doubt about the validity of the results of hit-metering a given set of resources, the server can employ cache-busting techniques for short periods, to establish a baseline for validating the hit-metering results. Various approaches to this problem are discussed in a paper by James Pitkow [9]. 📄

4.2 What about "Network Computers"? 💻

The analysis in section 4.1 assumed that "almost all Web users" have client caches. If the Network Computers (NC) model becomes popular, however, then this assumption may be faulty: most proposed NCs have no disk storage, and relatively little RAM. Many Personal Digital Assistants (PDAs), which sometimes have network access, have similar constraints. Such client systems may do little or no caching of HTTP responses. This means that a single user might well generate many unconditional GETs that yield the same response from a proxy cache. 🫣

First note that the hit-metering design in this document, even with such clients, provides an approximation no worse than available with unmodified HTTP/1.1: the counts that a proxy would return to an origin server would represent exactly the number of requests that the proxy would forward to the server, if the server simply specifies Cache-control: s-maxage=0. 🌈

However, it might be possible to improve the accuracy of these hit-counts by use of some heuristics at the proxy. For example, the proxy might note the IP address of the client, and count only one GET per client address per response. This is not perfect: for example, it fails to distinguish between NCs and certain other kinds of hosts. The proxy might also use the heuristic that only those clients that never send a conditional GET should be treated this way, although we are not at all certain that NCs will never send conditional GETs. 🤷‍♀️

Since the solution to this problem appears to require heuristics based on the actual behavior of NCs (or perhaps a new HTTP protocol feature that allows unambiguous detection of cacheless clients), it appears to be premature to specify a solution. 🫶

4.3 Critical-path delay analysis ⏱️

okay so latency is literally the main character here 💅 In systems like the Web, a bunch of steps depend on each other, so the final result can't happen until all its predecessors are done. The tree shape does allow some parallelism, so you can't just add up every step's timing to find the total latency 🙈 But any single path through that dependency tree can't be parallelized, and the longest one is what sets the overall latency in seconds. That's the "critical path" 🛤️ No matter how much you speed up any other path, the final result's latency doesn't budge.

If the final result for a Web request is a page rendering in a browser (or whatever acting on the response looks like), then some network round trips are on the critical path, like exchanging TCP SYN packets when there's no connection yet 🫶 This hit-metering design does add some round trips for reporting non-zero counts when a cache entry gets removed, BUT by design those are off the critical path ✨ They can run in parallel with any other operation and only need "best efforts," so a proxy doesn't have to line up other operations around whether they succeed or fail 🦋

Anything that changes network utilization, whether it goes up or down, can indirectly affect how fast things feel to the user 🌸 The expectation is that hit-metering, on average, reduces loading, so even its indirect effects shouldn't add network round trips to any critical path. Still, there might be a few specific cases where the extra off-critical-path work (specifically, usage reports sent when a cache entry is removed) ends up delaying an operation that is on the critical path 😭 That's an unavoidable problem in datagram networks.

5 Specification 📜

5.1 Specification of Meter header and directives 🎀

The Meter general-header field is used to:

  • Negotiate the use of hit-metering and usage-limiting among origin servers and proxy caches.
  • Report use counts and reuse counts.

Implementing the Meter header is optional for both proxies and origin servers 🌈 BUT, any proxy that sends the Meter header in a request MUST implement every requirement of this specification, no exceptions and no edits 💀

The Meter header MUST always be protected by a Connection header 🔒 A proxy that doesn't implement the Meter header MUST NOT pass it through to another system (see section 5.5 for how a non-caching proxy may comply with this specification). If a Meter header shows up in a message whose version is less than HTTP/1.1, it MUST be ignored, because it has clearly flowed through a proxy that doesn't implement Meter 🙅‍♀️

If a proxy gets a response with a version less than HTTP/1.1, that means it came from a server (or another proxy) that doesn't implement the Meter header. In that case the proxy SHOULD NOT send Meter request directives to that server, since they'd just waste bandwidth 🗑️ This doesn't apply if the proxy is currently hit-metering or usage-limiting any responses from that server. And if the proxy gets an HTTP/1.1 or higher response from such a server, it should stop suppressing the Meter directives ✨

All proxies sending the Meter header MUST adhere to the "metering subtree" design described in section 3 🌳

Meter = "Meter" ":" 0#meter-directive

meter-directive = meter-request-directive
                | meter-response-directive
                | meter-report-directive

meter-request-directive =
                  "will-report-and-limit"
                | "wont-report"
                | "wont-limit"

meter-report-directive =
                | "count" "=" 1*DIGIT "/" 1*DIGIT

meter-response-directive =
                  "max-uses" "=" 1*DIGIT
                | "max-reuses" "=" 1*DIGIT
                | "do-report"
                | "dont-report"
                | "timeout" "=" 1*DIGIT
                | "wont-ask"

A meter-request-directive or meter-report-directive may only appear in an HTTP request message 📨 A meter-response-directive may only appear in an HTTP response directive.

An empty Meter header in a request means "Meter: will-report-and-limit" 🍓 An empty Meter header in a response, or any other response with one or more Meter headers that don't include "dont-report" or "wont-ask", implies "Meter: do-report" 💜

The meaning of the meter-request-directives is as follows:

will-report-and-limit : indicates that the proxy is willing and able to return usage reports and will obey any usage-limits. 🫡

wont-report : indicates that the proxy will obey usage-limits but won't send usage reports. 🙃

wont-limit : indicates that the proxy won't obey usage-limits but will send usage reports. 📬

A proxy that's willing neither to obey usage-limits nor to send usage reports MUST NOT transmit a Meter header in the request 🚫

The meaning of the meter-report-directives is as follows:

count "=" 1*DIGIT "/" 1*DIGIT : Both digit strings encode decimal integers. The first integer is the count of uses of the cache entry since the last report. The second integer is the count of reuses of the entry. 🔢

Section 5.3 specifies the counting rules.

The meaning of the meter-response-directives is as follows:

max-uses "=" 1*DIGIT : sets an upper limit on the number of "uses" of the response, not counting its immediate forwarding to the requesting end-client, for all proxies in the following subtree taken together. 🧮

max-reuses "=" 1*DIGIT : sets an upper limit on the number of "reuses" of the response for all proxies in the following subtree taken together. 🔁

do-report : specifies that the proxy MUST send usage reports to the server. 📤

dont-report : specifies that the proxy SHOULD NOT send usage reports to the server. 🤐

timeout "=" 1*DIGIT : sets a metering timeout of the specified number of minutes (not seconds) after the origination of this response (as indicated by its "Date" header). If the proxy has a non-zero hit count for this response when the timeout expires, it MUST send a report to the server at or before that time. Timeouts should be implemented with an accuracy of plus or minus one minute. Implies "do-report". ⏰

wont-ask : specifies that the proxy SHOULD NOT send any Meter headers to the server. The proxy should forget this advice after a period of no more than 24 hours. 🙈

Section 5.3 specifies the counting rules, and in particular specifies a somewhat non-obvious interpretation of the max-uses value. 👀

5.2 Abbreviations for Meter directives ✂️

To allow for the most efficient possible encoding of Meter headers, abbreviated forms of all Meter directives are defined too 🦋 These are exactly semantically equivalent to their non-abbreviated counterparts. All systems implementing the Meter header MUST implement both the abbreviated and non-abbreviated forms. Implementations SHOULD use the abbreviated forms in normal use 💖

The abbreviated forms are shown below, with the matching non-abbreviated literals in the comments:

Abb-Meter = "Meter" ":" 0#abb-meter-directive

abb-meter-directive = abb-meter-request-directive
                | abb-meter-response-directive
                | abb-meter-report-directive

abb-meter-request-directive =
                  "w"           ; "will-report-and-limit"
                | "x"           ; "wont-report"
                | "y"           ; "wont-limit"

abb-meter-report-directive =
                | "c" "=" 1*DIGIT "/" 1*DIGIT   ; "count"

abb-meter-response-directive =
                  "u" "=" 1*DIGIT       ; "max-uses"
                | "r" "=" 1*DIGIT       ; "max-reuses"
                | "d"                   ; "do-report"
                | "e"                   ; "dont-report"
                | "t" "=" 1*DIGIT       ; "timeout"
                | "n"                   ; "wont-ask"

Note: even though the Abb-Meter BNF rule is defined separately from the Meter rule, you can freely mix abbreviated and non-abbreviated Meter directives in the same header 🧃

5.3 Counting rules 🧁

Note: remember that hit-counts and usage-counts are associated with individual responses, not with resources. A cache entry that holds more than one response over its lifetime is also not a "response," in this particular sense. 🧠

Let R be a cached response, and V be the value of the Request-URI and selecting request-headers (if any, see section 14.43 of the HTTP/1.1 specification [4]) that would select R if contained in a request. We define a "use" of R as occurring when the proxy returns its stored copy of R in a response with any of the following status codes: a 200 (OK) status; a 203 (Non-Authoritative Information) status; or a 206 (Partial Content) status when the response contains byte #0 of the entity (see section 5.4 for a discussion of Range requests). 📦

Note: when a proxy forwards a client's request and receives a response, the response that the proxy sends immediately to the requesting client is not counted as a "use". I.e., the reported count is the number of times the cache entry was used, and not the number of times that the response was used. 🙃

We define a "reuse" of R as as occurring when the proxy responds to a request selecting R with a 304 (Not Modified) status, unless that request is a Range request that does not specify byte #0 of the entity. 🔁

5.3.1 Counting rules for hit-metering 📊

A proxy participating in hit-metering for a cache response R maintains two counters, CU and CR, associated with R. When a proxy first stores R in its cache, it sets both CU and CR to 0 (zero). When a subsequent client request results in a "use" of R, the proxy increments CU. When a subsequent client request results in a "reuse" of R, the proxy increments CR. When a subsequent client request selecting R (i.e., including V) includes a "count" Meter directive, the proxy increments CU and CR using the corresponding values in the directive. 🧮✨

When the proxy sends a request selecting R (i.e., including V) to the inbound server, it includes a "count" Meter directive with the current CU and CR as the parameter values. If this request was caused by the proxy's receipt of a request from a client, then when the server's response arrives, the proxy sets CU and CR to the number of uses and reuses, respectively, that may have occurred while the request was in progress. (These numbers are likely, but not certain, to be zero.) If the proxy's request was a final HEAD-based report, it doesn't need to keep maintaining the CU and CR values, but it may also set them to the number of intervening uses and reuses and keep them 🫶

5.3.2 Counting rules for usage-limiting ✨

Okay bestie, so a proxy doing usage-limiting for a response R keeps either or both of two counters, TU and TR, depending on what that resource needs 💅. TU and TR go up in the exact same way as CU and CR do. The difference is the reset. TU only gets zeroed when a "max-uses" Meter directive comes in for that response (the very first one counts too), and TR only gets zeroed when a "max-reuses" Meter directive comes in for that response 🎀.

A proxy doing usage-limiting for R also stores MU and/or MR for R. Here's the vibe check:

  • If it gets a response with only a max-uses value, it sets MU to that value and MR to infinity.
  • If it gets a response with only a max-reuses value, it sets MR to that value and MU to infinity.
  • If it gets a response with both max-reuses and max-reuses values, it sets MU and MR to those values, respectively.
  • If it gets a later response with neither max-reuses nor max-reuses values, it sets both MU and MR to infinity 🌸.

Now the important part 🔥. If a proxy doing usage-limiting for R gets a request that would cause a "use" of R, and TU >= MU, it MUST forward the request to the server. If it gets a request that would cause a "reuse" of R, and TR >= MR, it MUST forward the request to the server. If either case happens and the proxy has already sent a previous request to the server and is waiting on the response, it should hold off on handling the new request until the response arrives (or times out). It SHOULD NOT have two revalidation requests pending at once that select the same response, unless these are Range requests selecting different subranges 🫶.

There's a special case for "max-uses" that's lowkey tricky 🙈. If the proxy gets a response with "max-uses=0" and doesn't forward it to a requesting client, the proxy should set a flag PF associated with R. If R is true, then when a request arrives while if TU >= MU, if the PF flag is set, then the request need not be forwarded to the server (provided that this is not required by other caching rules). However, the PF flag MUST be cleared on any use of the response 💜.

Note: the "PF" flag is so named because this feature is useful only for caches that could issue a "prefetch" request before an actual client request for the response. A proxy not implementing prefetching need not implement the PF flag.

5.3.3 Equivalent algorithms are allowed 🦋

Any other algorithm that exhibits the same external behavior (i.e., generates exactly the same requests from the proxy to the server) as the one in this section is explicitly allowed. Totally fair, as long as the server sees the same requests ✨.

Note: in most cases, TU will be equal to CU, and TR will be equal to CR. The only two cases where they could differ are: 1. The proxy issues a non-conditional request for the resource using V, while TU and/or TR are non-zero, and the server's response includes a new "max-uses" and/or "max-reuses" directive (thus zeroing TU and/or TR, but not CU and CR). 2. The proxy issues a conditional request reporting the hit-counts (and thus zeroing CU and CR, but not TU or TR), but the server's response does not include a new "max-uses" and/or "max-reuses" directive.

To handle the first case, the proxy has a few implementation options 🍓:

  • Always store TU and TR separately from CU and CR.
  • Create "shadow" copies of TU and TR when this situation arises (analogous to "copy on write").
  • Generate a HEAD-based usage report when the non-conditional request is sent (or when the "max-uses=0" is received), causing CU and CR to be zeroed (analogous in some ways to a "memory barrier" instruction).

In the second case, the server has implicitly removed the usage-limit(s) on the response (by setting MU and/or MR to infinity), so the fact that, say, TU is different from CU is not significant 🌈.

Note: It may also be possible to eliminate the PF flag by sending extra HEAD-based usage-report requests, but we recommend against this; it is better to allocate an extra bit per entry than to transmit extra requests.

5.4 Counting rules: interaction with Range requests 🧚

HTTP/1.1 lets a client request sub-ranges of a resource. A client might end up sending several requests that together result in one copy of the resource. For the results seen by origin servers to be uniform, proxies need to follow a rule for counting these references, even though it's not clear that one rule gets accurate results in every case 🤷‍♀️.

The rule in this specification is that proxies count as a "use" or "reuse" only those Range requests that result in the return of byte #0 of the resource. The reasoning is that in almost every case an end-client will retrieve the beginning of any resource it references at all, and it will seldom retrieve any portion more than once. So this rule appears to hit the goal of a "best-efforts" approximation 👑.

5.5 Implementation by non-caching proxies 🌸

A non-caching proxy may join the metering subtree, and this is strongly recommended 💖.

A non-caching proxy (HTTP/1.1 or higher) that joins the metering subtree SHOULD forward Meter headers on both requests and responses, with the appropriate Connection headers ✨.

If a non-caching proxy forwards Meter headers, it MUST comply with these restrictions 🔥:

  1. If the proxy forwards Meter headers in responses, such a response MUST NOT be returned to any request except the one that elicited it.
  2. Once a non-caching proxy starts forwarding Meter headers, it should not arbitrarily stop forwarding them (or else reports may be lost).

A proxy that caches some responses and not others, for whatever reason, may choose to implement the Meter header as a caching proxy for the responses it caches, and as a non-caching proxy for the responses it doesn't cache, as long as its external behavior with respect to any particularly response is fully consistent with this specification 🦋.

5.6 Implementation by cooperating caches 🫶

Several HTTP cache implementations, most notably the Harvest/Squid cache [2], set up cooperative arrangements between several caches. If such caches use a protocol other than HTTP to talk to each other, such as the Internet Cache Protocol (ICP) [12], and if they implement the Meter header, then they MUST act to ensure that their cooperation does not violate the intention of this specification 🙌.

Specifically, if one member of a group of cooperating caches agrees with a server to hit-meter a particular response, and then passes this response via a non-HTTP protocol to a second cache in the group, the caches MUST ensure that the server which requested the metering receives reports that appropriately account for any uses or resues made by the second cache. Similarly, if the first cache agreed to usage-limit the response, the total number of uses by the group of caches MUST be limited to the agreed-upon number 💜.

6 Examples 🎀

6.1 Example of a complete set of exchanges 💅

This example shows how the protocol is meant to be used most of the time: for hit-metering without usage-limiting. Entity bodies are left out 🌈.

A client sends a request to a proxy:

GET http://foo.com/bar.html HTTP/1.1

The proxy forwards the request to the origin server:

GET /bar.html HTTP/1.1
Host: foo.com
Connection: Meter

So it's offering (implicitly) "will-report-and-limit" 🎁.

The server responds to the proxy:

HTTP/1.1 200 OK
Date: Fri, 06 Dec 1996 18:44:29 GMT
Cache-control: max-age=3600
Connection: meter
Etag: "abcde"

So it's (implicitly) requiring "do-report" (but not requiring usage-limiting) 👀.

The proxy responds to the client:

HTTP/1.1 200 OK
Date: Fri, 06 Dec 1996 18:44:29 GMT
Etag: "abcde"
Cache-control: max-age=3600, proxy-mustcheck
Age: 1

Since the proxy doesn't know if its client is an end-system or a proxy that doesn't do metering, it adds the "proxy-mustcheck" directive 🧚.

Another client soon asks for the resource:

GET http://foo.com/bar.html HTTP/1.1

and the proxy sends the same response it sent to the other client, except (perhaps) for the Age value ✨.

After an hour has passed, a third client asks for the response:

GET http://foo.com/bar.html HTTP/1.1

But now the response's max-age has been exceeded, so the proxy revalidates the response with the origin server:

GET /bar.html HTTP/1.1
If-None-Match: "abcde"
Host: foo.com
Connection: Meter
Meter: count=1/0

This one request does double duty 🫶: it fulfills the proxy's duty to validate the response and reports the one "use" that wasn't forwarded.

The origin server responds:

HTTP/1.1 304 Not Modified
Date: Fri, 06 Dec 1996 19:44:29 GMT
Cache-control: max-age=3600
Etag: "abcde"

So the proxy can use the original response to reply to the new client, and it also zeros the use-count it associates with that response 🌸.

Another client soon asks for the resource:

GET http://foo.com/bar.html HTTP/1.1

and the proxy sends the appropriate response 💖.

After another few hours, the proxy decides to remove the cache entry. When it does, it sends this to the origin server:

HEAD /bar.html HTTP/1.1
If-None-Match: "abcde"
Host: foo.com
Connection: Meter
Meter: count=1/0

reporting that one more use of the response was satisfied from the cache 🍓.

6.2 Protecting against HTTP/1.0 proxies 🛡️

An origin server that doesn't want HTTP/1.0 caches to store the response at all, and is willing to have HTTP/1.0 end-system clients generate excess GETs (which will be forwarded by HTTP/1.0 proxies), could send this as its reply:

HTTP/1.1 200 OK
Cache-control: max-age=3600
Connection: meter
Etag: "abcde"
Expires: Sun, 06 Nov 1994 08:49:37 GMT

HTTP/1.0 caches will see the ancient Expires header, but HTTP/1.1 caches will see the max-age directive and ignore Expires 💅.

Note: although most major HTTP/1.0 proxy implementations observe the Expires header, it is possible that some in use do not. Use of the Expires header to prevent caching by HTTP/1.0 proxies might not be entirely reliable.

6.3 More elaborate examples 🌈

Here's a request from a proxy that's willing to hit-meter but isn't willing to usage-limit:

GET /bar.html HTTP/1.1
Host: foo.com
Connection: Meter
Meter: wont-limit

Here's a response from an origin server that doesn't want hit counting, but does want "uses" limited to 3, and "reuses" limited to 6 👑:

HTTP/1.1 200 OK
Cache-control: max-age=3600
Connection: meter
Etag: "abcde"
Expires: Sun, 06 Nov 1994 08:49:37 GMT
Meter: max-uses=3, max-reuses=6, dont-report

And here's the same example with abbreviated Meter directive names 🦋:

HTTP/1.1 200 OK
Cache-control: max-age=3600
Connection: meter
Etag: "abcde"
Expires: Sun, 06 Nov 1994 08:49:37 GMT
Meter:u=3,r=6,e

7. Interactions with content negotiation ✨

okay bestie so this part is about where hit-metering meets "content-negotiated" resources (the ones that come in different flavors depending on what the client asks for) 🎀 there are two things to know:

  1. treatment of responses carrying a Vary header (section 7.1).
  2. treatment of responses that use the proposed Transparent Content Negotiation mechanism (section 7.2).

7.1 Treatment of responses carrying a Vary header 💅

okay so separate counts should be kept for each combination of the headers named in the Vary header for the Request-URI (what [4] calls "the selecting request-headers"), even if they map to the same entity-tag 🧚 this means hits get counted on each variant if a page has multiple variants available 🫶

Note: This interaction between Vary and the hit-counting directives allows the origin server a lot of flexibility in specifying how hits should be counted. In essence, the origin server uses the Vary mechanism to divide the requests for a resource into arbitrary categories, based on the request- headers. (We will call these categories "request-patterns".) Since a proxy keeps its hit-counts for each request-pattern, rather than for each resource, the origin server can obtain separate statistics for many aspects of an HTTP request. 🦋

for example, if a page varied based on the value of the User-Agent header in the requests, then hit counts would be kept for each different flavor of browser 🌈 but it's honestly more general than that 😭 because multiple header combinations can map to the same variant, it also lets the origin server count the number of times (e.g.) the Swahili version of a page was requested, even though it's only available in English 🌸

if a proxy does not support the Vary mechanism, then [4] says that it MUST NOT cache any response that carries a Vary header, and hence need not implement any aspect of this hit-counting or usage-limiting design for varying resources 💀

Note: this also implies that if a proxy supports the Vary mechanism but is not willing to maintain independent hit-counts for each variant response in its cache, then it must follow at least one of these rules: 1. It must not use the Meter header in a request to offer to hit-meter or usage-limit responses. 2. If it does offer to hit-meter or usage-limit responses, and then receives a response that includes both a Vary header and a Meter header with a directive that it cannot satisfy, then the proxy must not cache the response.

so a proxy is allowed to partially implement the Vary mechanism with respect to hit-metering, as long as this has no externally visible effect on its ability to comply with the Meter specification 🔥 that's a whole vibe honestly 👑

this approach works for counting almost any aspect of the request stream, without embedding any specific list of countable aspects in the specification or proxy implementation 🍓

7.2 Interaction with Transparent Content Negotiation

[A description of the interaction between this design and the proposed Transparent Content Negotiation (TCN) design [6] will be made available in a later document.] 🙈

8. A Note on Capturing Referrals 🌸

okay so it's alleged that some advertisers want to pay content providers, not by the "hit", but by the "nibble" 🍓 that's the number of people who actually click on the ad to get more info 😭

now, HTTP already has a mechanism for doing this: the "Referer" header 🎀 however, maybe it ought to be disabled for privacy reasons, according to the HTTP/1.1 spec 🙈

"Because the source of the link may be private information or may reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent."

however, in the case of ads, the source of the link actually wants to let the referred-to page know where the reference came from 💖

this doesn't need any extra mechanism 💅 it can just use schemes that embed the referrer in the URI, kind of like this:

http://www.blah.com/ad-reference?from=site1

a URI like that should point to a resource (perhaps a CGI script) which returns a 302 redirect to the real page:

http://www.blah.com/ad-reference.html

proxies which do not cache 302s will cause one hit on the redirection page per use, but the real page will get cached ✨ proxies which do cache 302s and report hits on the cached 302s will behave optimally 🦋

this approach has the advantage that it works whether or not the end-client has disabled the use of Referer 🫶 combined with the rest of the hit-metering proposal in this design, it lets an advertiser know how often a reference to an advertisement was made from a particular page 👑

9. Alternative proposals 🧚

there might be a bunch of other ways to gather demographic and usage info 🌈 other mechanisms might respond to a different set of needs than this proposal does 💜 this proposal certainly does not preclude the proposal or deployment of other such mechanisms, and many of them may be complementary to and compatible with the mechanism proposed here ⭐

there's been some speculation that statistical sampling methods might be used to gather reasonably accurate data 👀 one such proposal is to manipulate cache expiration times so that selected resources are uncachable for carefully chosen periods, letting servers accurately count accesses during those periods 🔥 the hit-metering mechanism proposed here is entirely complementary to that approach, since it could be used to reduce the cost of gathering those counts 💖 James Pitkow has written a paper comparing an earlier draft of this hit-metering proposal with sampling approaches [9] 📚

Phillip Hallam-Baker has proposed using a log-exchange protocol [5], by which a server could request a proxy's logs by making an HTTP request to the proxy 🫠 this proposal asserts that it is "believed to operate correctly in configurations involving multiple proxies", but it's not clear that this is true if an outer proxy is used as a (one-way) firewall 😬 the proposal also leaves a number of open issues, such as how an origin server can be sure that all of the proxies in the request subtree actually support log-exchange 💀 it's also not clear how this proposal couples a proxy's support of log-exchange to a server's permission to cache a response 🙃

for general background on Web measurement standards, see the discussion by Thomas P. Novak and Donna L. Hoffman [8] 🍓 also see the "Privacy and Demographics Overview" page maintained by by the World Wide Web Consortium [10], which includes a pointer to some tentative proposals for gathering consumer demographics (not just counting references) [3] 🎀

10. Security Considerations 🔥

okay bestie this one's serious, so let's go 😤

which outbound clients should a server (proxy or origin) trust to report hit counts? 👀 a malicious proxy could easily report a large number of hits on some page, and thus perhaps cause a large payment to a content provider from an advertiser 💸 to help avoid this, a proxy may choose to only relay usage counts received from its outbound proxies to its inbound servers when the proxies have authenticated themselves using Proxy-Authorization and/or they are on a list of approved proxies 🔒

it is not possible to enforce usage limits if a proxy is willing to cheat (i.e., it offers to limit usage but then ignores a server's Meter directive) 😭🚫

regarding privacy: it appears that the design in this document does not reveal any more information about individual users than would already be revealed by implementation of the existing HTTP/1.1 support for "Cache-control: max-age=0, proxy-revalidate" or "Cache-control: s-maxage=0" 🙈 it may, in fact, help to conceal certain aspects of the organizational structure on the outbound side of a proxy 🫶 in any case, the conflict between user requirements for anonymity and origin server requirements for demographic information cannot be resolved by purely technical means 💜

11. Acknowledgments 💖

we gratefully acknowledge the constructive comments received from Anselm Baird-Smith, Ted Hardie, Koen Holtman (who suggested the technique described in section 8), Dave Kristol, Ari Luotonen, Patrick R. McManus, Ingrid Melve, and James Pitkow 🫶✨ thank you all, you're literally the best 👑

12. References 📚

  1. Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  2. Anwat Chankhunthod, Peter B. Danzig, Chuck Neerdaels, Michael F. Schwartz, and Kurt J. Worrell. A Hierarchical Internet Object Cache. Proc. 1996 USENIX Technical Conf., San Diego, January, 1996, pp. 153-163.
  3. Daniel W. Connolly. Proposals for Gathering Consumer Demographics. http://www.w3.org/pub/WWW/Demographics/Proposals.html.
  4. Fielding, R., Gettys, J., Mogul, J., Nielsen, H. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1," RFC 2068, January, 1997.
  5. Phillip M. Hallam-Baker. Notification for Proxy Caches. W3C Working Draft WD-proxy-960221, World Wide Web Consortium, February, 1996. http://www.w3.org/pub/WWW/TR/WD-proxy.html.
  6. Holtman, K., and A. Mutz, "Transparent Content Negotiation in HTTP", Work in Progress.
  7. Mogul, J., "Forcing HTTP/1.1 proxies to revalidate responses", Work in Progress.
  8. Thomas P. Novak and Donna L. Hoffman. New Metrics for New Media: Toward the Development of Web Measurement Standards. This is a draft paper, currently available at http://www2000.ogsm.vanderbilt.edu/novak/web.standards/webstand.html. Cited by permission of the author; do not quote or cite without permission.
  9. James Pitkow. In search of reliable usage data on the WWW. Proc. Sixth International World Wide Web Conference, Santa Clara, CA, April, 1997.
  10. Joseph Reagle, Rohit Khare, Dan Connolly, and Tim Berners-Lee. Privacy and Demographics Overview. http://www.w3.org/pub/WWW/Demographics/.
  11. Linda Tauscher and Saul Greenberg. Revisitation Patterns in World Wide Web Navigation. Research Report 96/587/07, Department of Computer Science, University of Calgary, March, 1996. http://www.cpsc.ucalgary.ca/projects/grouplab/papers/96WebReuse/TechReport96.html.
  12. Wessels, D., and K. Claffy "Internet Cache Protocol (ICP), version 2", RFC 2186, September 1997.

13. Authors' Addresses 🎀

Jeffrey C. Mogul Western Research Laboratory Digital Equipment Corporation 250 University Avenue Palo Alto, California, 94305, U.S.A.

EMail: mogul@wrl.dec.com Phone: 1 415 617 3304 (email preferred)

Paul J. Leach Microsoft 1 Microsoft Way Redmond, Washington, 98052, U.S.A.

EMail: paulle@microsoft.com

Okay bestie, copyright time 🎀 Copyright (C) The Internet Society (1997). All Rights Reserved. 💅

So here's the vibe: you can copy this document and share it with others 🦋 You can also make derivative works that comment on or explain it, or help people implement it 🌸 Those works can be prepared, copied, published, and distributed, in whole or in part, with no restrictions of any kind, as long as the copyright notice and this paragraph are included on every copy and derivative work. 💖

BUT (lowkey important) the document itself can't be modified in any way. You can't remove the copyright notice or the references to the Internet Society or other Internet organizations. The exceptions are developing Internet standards, where the copyright procedures in the Internet Standards process must be followed, or translating it into languages other than English 🌍

The limited permissions above are perpetual and won't be revoked by the Internet Society or its successors or assigns. Forever and ever 🫶✨

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 💀

Translation: no warranties, no promises, just vibes 🙈
📦 where this page comes from