RFC 4236

HTTP Adaptation with Open Pluggable Edge Services (OPES)

📅 published November 2005, from ietf

RFC 4236: HTTP Adaptation with Open Pluggable Edge Services (OPES) 🎀

okay bestie, let's get into it 💅 this one is from November 2005 and it's a Standards Track doc, so it's a real protocol spec, not just a vibe. The authors are A. Rousskov (The Measurement Factory) and M. Stecher (CyberGuard Corporation). The header doesn't list anything it obsoletes or updates, so no drama there 🫶

Status of This Memo

This doc specifies an Internet standards track protocol for the Internet community and asks for discussion and suggestions for improvements ✨ For the standardization state and status of this protocol, check the current edition of "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🌸

Copyright (C) The Internet Society (2005) 🦋

Abstract

Open Pluggable Edge Services (OPES) framework documents several application-agnostic mechanisms, like OPES tracing, OPES bypass, and the OPES callout protocol 🧚 This doc extends those generic mechanisms for Hypertext Transfer Protocol (HTTP) adaptation. Together, the application-agnostic OPES docs and this HTTP profile make up a complete spec for HTTP adaptation with OPES 💖

Table of Contents

   1. Scope ...........................................................3
   2. OPES Document Map ...............................................3
   3. Callout Protocol ................................................4
      3.1. Application Message Parts ..................................5
      3.2. Application Profile Features ...............................6
           3.2.1. Profile Parts .......................................6
           3.2.2. Profile Structure ...................................8
           3.2.3. Aux-Parts ...........................................8
           3.2.4. Pause-At-Body .......................................9
           3.2.5. Stop-Receiving-Body ................................10
           3.2.6. Preservation-Interest-Body .........................10
           3.2.7. Content-Encodings ..................................11
           3.2.8. Profile Negotiation Example ........................12
      3.3. Application Message Start Message .........................13
      3.4. DUM Message ...............................................13
      3.5. Selective Adaptation ......................................14
      3.6. Hop-by-hop Headers ........................................15
      3.7. Transfer Encodings ........................................15
      3.8. HTTP Header Correctness ...................................16
           3.8.1. Message Size Recalculation .........................16
           3.8.2. Content-MD5 Header .................................17
      3.9. Examples ..................................................18
   4. Tracing ........................................................22
   5. Bypass .........................................................24
   6. IAB Considerations .............................................24
   7. Security Considerations ........................................24
   8. IANA Considerations ............................................24
   9. Compliance .....................................................25
   10. References ....................................................25
      10.1. Normative References .....................................25
      10.2. Informative References ...................................25

1. Scope ✨

OPES framework docs cover several app-agnostic mechanisms, like OPES processor and endpoint communications [RFC3897] or the OPES callout protocol [RFC4037] 💅 This doc extends those generic mechanisms so they work for one specific app protocol, HTTP [RFC2616]. Together, the app-agnostic OPES docs and this HTTP profile are a complete spec for HTTP adaptation with OPES 🫶

The main sections of this doc are the HTTP-specific extensions for the matching app-agnostic mechanisms, which are documented elsewhere 🦋

2. OPES Document Map 🗺️

This doc is part of a big set of OPES specs made by the IETF OPES Working Group. Knowing the overall OPES approach and typical scenarios is often essential if you want to understand any single OPES doc on its own 🧠 So this section is basically an index to help you find the "missing" info 🙈

  • The "OPES Use Cases and Deployment Scenarios" doc [RFC3752] describes the services and apps that are in scope for OPES. It was the motivation and guidance for designing the OPES architecture 🌈
  • The OPES architecture and common terminology are in "An Architecture for Open Pluggable Edge Services (OPES)" [RFC3835] 💜
  • "Policy, Authorization and Enforcement Requirements of OPES" [RFC3838] lays out requirements and assumptions for the policy framework, but doesn't pin down specific authorization and enforcement methods 🎀
  • "Security Threats and Risks for OPES" [RFC3837] is the OPES risk analysis, and it doesn't recommend specific solutions 😭
  • "OPES Treatment of IAB Considerations" [RFC3914] covers all the architecture-level considerations the IETF Internet Architecture Board (IAB) raised when the OPES WG was chartered 👑
  • At the core of OPES are the OPES processor and the callout server. These two network elements talk to each other over an OPES Callout Protocol (OCP) 🔥 The requirements for that protocol are discussed in "Requirements for OPES Callout Protocols" [RFC3836] ✨
  • "OPES Callout Protocol Core" [RFC4037] specifies an app-agnostic protocol core for communication between the OPES processor and the callout server 🧚
  • "OPES entities and end points communications" [RFC3897] specifies the generic tracing and bypass mechanisms for OPES 🌸
  • The OCP Core and Communications docs don't depend on the app protocol being adapted. Their generic mechanisms have to be paired with app-specific profiles. This doc, HTTP adaptation with OPES, is that profile for HTTP. It specifies how the app-agnostic OPES mechanisms get used and extended so HTTP messages can be adapted 💖
  • Finally, "P: Message Processing Language" [rules-p] defines a language for specifying which OPES adaptations (e.g., translation) get applied to which app messages (e.g., e-mail from bob@example.com). The P language is meant for configuring application proxies (OPES processors) 💅

3. Callout Protocol 📞

This section documents the HTTP profile for the OPES Callout Protocol (OCP) Core [RFC4037]. You need to be familiar with OCP Core to understand the HTTP profile, and this section uses OCP Core's conventions, terminology, and mechanisms 🧠

The OPES processor says it wants to adapt HTTP messages with a Negotiation Offer (NO) message that includes HTTP-specific feature identifiers, documented in Section 3.2. HTTP-specific OCP optimization mechanisms can get negotiated at the same time too ✨ A callout server that supports HTTP message adaptation gets a chance to negotiate which HTTP message parts will take part in the adaptation. That includes negotiating HTTP request parts as metadata for HTTP response adaptation. The negotiable HTTP message parts are documented in Section 3.1 🫶

The HTTP profile adds a new parameter to the Application Message Start (AMS) message so the known HTTP message length can be communicated. HTTP headers often don't convey length info reliably, or at all, so this helps 🦋 That parameter is documented in Section 3.3. Section 3.4 documents a way to report HTTP message parts with Data Use Mine (DUM) messages 💜

The remaining OCP sections cover various OCP marshaling corner cases, like handling HTTP transfer encodings and 100 Continue responses 🌈

3.1. Application Message Parts 🎀

An HTTP message can have several well-known parts: headers, body, and trailers 🌸 OPES processors usually know a lot about these parts, because they have to isolate and interpret HTTP headers and find message boundaries. Callout servers might not care about certain parts, or they might benefit from reusing the OPES processor's work of isolating and categorizing the interesting ones 🧚

Here's the declaration of the am-part (application message part) type, using the OCP Core Protocol Element Type Declaration Mnemonic (PETDM):

am-part:  extends atom;
am-parts: extends list of am-part;

Figure 1

There are six valid am-part atoms 🍓

  • request-header: The start-line of an HTTP request message, all request message headers, and the CRLF separator at the end of the HTTP headers (compare with section 4.1 of [RFC2616]) 💅
  • request-body: The message body of an HTTP request message, as defined in section 4.3 of [RFC2616], but not including the trailer 🫶
  • request-trailer: The entity headers of the trailer of an HTTP request message in chunked transfer encoding. This part follows the same syntax as the trailer defined in section 3.6.1 of [RFC2616] 🦋
  • response-header: The start-line of an HTTP response message, all response message headers, and the CRLF separator at the end of the HTTP headers (compare with section 4.1 of [RFC2616]) 💜
  • response-body: The message body of an HTTP response message, as defined in section 4.3 of [RFC2616], but not including the trailer 🌈
  • response-trailer: The entity headers of the trailer of an HTTP response message in chunked transfer encoding. This part follows the same syntax as the trailer defined in section 3.6.1 of [RFC2616] 🎀

3.2. Application Profile Features 🧚

This doc defines two HTTP profiles for OCP: a request profile and a response profile. Each one has a unique feature identifier, a list of original app message parts, and a list of adapted app message parts 💖

The profile ID for the request profile is http://www.iana.org/assignments/opes/ocp/http/request 💅

  • Original request parts: request-header, request-body, request-trailer ✨
  • Adapted request parts: request-header, request-body, request-trailer 🫶
  • Adapted response parts: response-header, response-body, response-trailer 🦋

The profile ID for the response profile is http://www.iana.org/assignments/opes/ocp/http/response 💜

  • Original transaction parts: request-header (aux), request-body (aux), request-trailer (aux), response-header, response-body, response-trailer 🌈
  • Adapted response parts: response-header, response-body, response-trailer 🎀

The request profile has two variants of adapted part lists: HTTP request parts and HTTP response parts 🧠 Parts with an "(aux)" suffix are auxiliary parts. They can only be used if they've been explicitly negotiated for a profile. Section 3.2.1 has the specific rules for negotiating and using am-parts 🔥

The scope of a negotiated profile is the OCP connection (the default), or the service group specified via the SG parameter 🌸

3.2.1. Profile Parts 🎀

okay so the rule is that an OCP agent MUST send application message parts in the order implied by the profile parts lists above 📋. if an OCP agent gets a part that's out of order, it MAY end the transaction with an error 💅

an OPES processor MUST NOT send parts that aren't listed as "original" in the negotiated profile 🙅‍♀️. a callout server MUST NOT send parts that aren't listed as "adapted" in the negotiated profile. if an OCP agent receives an not-listed part, it MUST terminate the transaction with an error 💀. the vibe behind that last one is to cut down on sneaky interop drama, where one agent thinks the other is using its parts when they're actually getting ignored or skipped because nobody was expecting them 😭

some HTTP messages are missing certain parts ✨. like, a lot of HTTP requests don't have bodies, and most HTTP messages don't have trailers 🙈. an OCP agent MUST NOT send (i.e., must skip) absent application message parts 🫶

an OCP agent MUST send the present non-auxiliary parts, and it MUST also send the present auxiliary parts that were negotiated via the Aux-Parts (Section 3.2.3) parameter 🌸. OCP agents MUST NOT send auxiliary parts that weren't negotiated via the Aux-Parts (Section 3.2.3) parameter 🚫

if an OCP agent gets a message part that breaks any of the stuff above, it MAY terminate that transaction with an error 😤

here's the deal: original parts that aren't on the adapted parts list can't be adapted 🧚. so a callout service can only adapt the parts on the adapted parts list, even if it can see other parts too 👀

in the request profile, the callout server MUST send either adapted request parts or adapted response parts 💖. if an OPES processor gets adapted flow with parts from both lists (which breaks the rule right above), it MUST terminate the OCP transaction with an error 🚨. basically, the callout server sends adapted response parts to "short-circuit" the HTTP transaction, so the OPES processor sends back an HTTP response without forwarding an adapted HTTP request 🦋. this is super useful for responding to something like an HTTP request the callout service has marked as forbidden 🙅‍♀️🔥

unless it's explicitly configured to do otherwise, an OPES processor MUST offer all non-auxiliary original parts in Negotiation Offer (NO) messages 👑. see Section 3.5 for why this rule exists and for examples of the nasty side effects you get from selective adaptation 💅

3.2.2. Profile Structure 🌈

an HTTP application profile feature extends the semantics of the feature type from OCP Core, and it adds these named parameters to that type ✨:

  • Aux-Parts (Section 3.2.3)
  • Pause-At-Body (Section 3.2.4)
  • Stop-Receiving-Body (Section 3.2.5)
  • Preservation-Interest-Body (Section 3.2.6)
  • Content-Encodings (Section 3.2.7)

here's how the HTTP profile feature structure looks when you write it with PETDM 🧩:

HTTP-Profile: extends Feature with {
    [Aux-Parts: am-parts];
    [Pause-At-Body: size];
    [Stop-Receiving-Body: size];
    [Preservation-Interest-Body: size];
    [Content-Encodings: codings];
};

Figure 2

an HTTP profile structure can go in the feature lists of Negotiation Offer (NO) messages, and it can also be an anonymous parameter of a Negotiation Response (NR) message 💜. every profile parameter applies to any OCP transaction inside the profile scope 🎀

3.2.3. Aux-Parts 🦋

the Aux-Parts parameter of an HTTP response profile is how you negotiate bringing auxiliary application message parts into the original data flow 🌸. its value is a list of am-part tokens that can be empty 💅. an OPES processor MAY send an Aux-Parts parameter to say which auxiliary application message parts it has available ✨. the callout server MAY reply with a subset of the parts it needs, and that subset can be empty too 🙈. whatever the callout server sends back is the subset of auxiliary message parts that actually got negotiated 🫶

when a callout server gets a Negotiation Offer (NO) message, it MUST ignore any non-auxiliary part listed in the Aux-Parts parameter 🙅‍♀️. when it sends a Negotiation Response (NR) message, the callout server MUST NOT select any application message part that wasn't explicitly listed in the negotiation offer 😤. if that last rule gets broken, the OPES processor MUST terminate the transaction 💀

an OPES processor MUST send each negotiated auxiliary part to the callout server, unless that part is absent 👑

example:

Aux-Parts: (request-header,request-body)

Figure 3

3.2.4. Pause-At-Body 💤

a callout server MAY use the Pause-At-Body parameter to ask for a pause in original application message body transmission before the original dataflow starts 🛑. its value is of type "offset" 🧁. the parameter marks where the start of the non-auxiliary application message body suffix is, which is the part the sender is temporarily not interested in seeing 🙈

[headers][ body prefix | body suffix ][trailer]
<-- ? --><-- offset  --><-- ? ---------------->
<-- equiv. DWP offset ->

Figure 4

when an OPES processor gets a Pause-At-Body parameter, it MUST behave as if it received a Want Data Paused (DWP) message with the corresponding org-offset 💖. the org-offset is different from the Pause-At-Body offset, and you won't know it until you know how big the HTTP message headers are 🧠

for example, if the Pause-At-Body value is zero, the OPES processor should send a Paused My Data (DPM) message just before it sends the first Data Use Mine (DUM) message with the response-body part in the HTTP response profile 🌷. if the Pause-At-Body value is 300, the OPES processor should send a DPM message after it has transmitted 300 OCTETs for that application message part ✨

example:

Pause-At-Body: 0

Figure 5

3.2.5. Stop-Receiving-Body 🛑

a callout server MAY use the Stop-Receiving-Body parameter to imply Want Stop Receiving Data (DWSR) message behavior before the original dataflow starts 🧚. its value is of type "offset" 🎀. the parameter points to an offset into the original, non-auxiliary message body part (request-body in the request profile and response-body in the response profile) 🍓

a callout service MAY send a Stop-Receiving-Body parameter with its negotiation response if there's a fixed offset into the message body for all transactions of a profile where a Want Stop Receiving Data (DWSR) message would be sent 💜. an OPES processor MUST behave as if it received a DWSR message with the corresponding offset. again, that offset is different from the Stop-Receiving-Body offset, and it isn't known until the HTTP message headers' size is known 🙃

for example, if the Stop-Receiving-Body value is zero in an HTTP response profile, the OPES processor should send an Application Message End (AME) message with result code 206 right after sending the response-header message part, and before it starts the response-body message part 🌸

example:

Stop-Receiving-Body: 0

Figure 6

3.2.6. Preservation-Interest-Body 🫧

the Preservation-Interest-Body parameter can be used to optimize data preservation at the OPES processor ✨. its value is of type "size", and it stands for a prefix size of the original, non-auxiliary message body part (request-body in the HTTP request profile and response-body in the response profile) 💅

a callout service MAY send a Preservation-Interest-Body parameter with its negotiation response if there's a fixed-size prefix of the application message body that would get a Data Preservation Interest (DPI) message 🦋. an OPES processor MUST behave as if it receives a DPI message with org-offset zero and org-size equal to the value of the Preservation-Interest-Body parameter 👑

for example, if the Preservation-Interest-Body value is zero in an HTTP response profile, the callout server must not send any Data Use Yours (DUY) message for the response-body part. the OPES processor can use that info to optimize its data preservation behavior even before it decides whether to preserve anything 🧠

example:

Preservation-Interest-Body: 0

Figure 7

3.2.7. Content-Encodings 🍓

a callout server MAY send a Content-Encodings list to show its preferences for content encodings 🎀. encodings listed first are preferred over the other encodings 💖. an OPES processor MAY use any content encoding when it sends application messages to a callout server ✨

the list of preferred content encodings doesn't mean the callout server doesn't support other encodings 🙈. an OPES processor MUST NOT bypass a service just because the actual content encoding doesn't match that service's preferences 😤

if an OCP agent gets an application message it can't handle because of a specific content encoding, the usual transaction termination rules apply 💀

grammar:

content-coding: extends atom;
content-codings: extends list of content-coding;

example:

Content-Encodings: (gzip)

Figure 8

the semantics of content-coding are defined in section 3.5 of [RFC2616] 📚

3.2.8. Profile Negotiation Example 🧁

example:

P: NO ({"54:http://www.iana.org/assignments/opes/ocp/http/response"
   Aux-Parts: (request-header,request-body)
   })
   SG: 5
   ;
S: NR {"54:http://www.iana.org/assignments/opes/ocp/http/response"
   Aux-Parts: (request-header)
   Pause-At-Body: 30
   Preservation-Interest-Body: 0
   Content-Encodings: (gzip)
   }
   SG: 5
   ;

Figure 9

this example shows a negotiation offer from an OPES processor for a service group (id 5) that has already been created, and the callout server sends back a suitable negotiation response 🫶

the OPES processor offers one profile feature for HTTP response messages 💅. on top of the standard message parts, the OPES processor can add the header and body of the original HTTP request as auxiliary message parts ✨

the callout server asks for the auxiliary request-header part, but it isn't interested in getting the request-body part 🙅‍♀️

the OPES processor sends at most these message parts, in this order, for all transactions in service group 5: request-header, response-header, response-body, response-trailer 🌈. the request-body part isn't included, because it's an auxiliary part that wasn't explicitly requested 🙃. some of the response parts might not get sent if the original message doesn't have them 💜

the callout server uses the Preservation-Interest-Body parameter with size zero to say it won't send any DUY messages 🧠. so the OPES processor doesn't have to preserve anything for any transaction in this profile 🫧

by sending a Pause-At-Body value of 30, the callout server asks for a data pause 🛑. the OPES processor sends a Paused My Data (DPM) message right after sending at least 30 OCTETs of the response-body part. after that, it waits for a Want More Data (DWM) message from the callout service 🦋

3.3. Application Message Start Message ✨

okay so a new named parameter for Application Message Start (AMS) messages is introduced 🎀 and it's called AM-EL 💖

AM-EL: size

Figure 10

AM-EL value is the size of the request-body part in the HTTP request profile, and is the size of the response-body part in the HTTP response profile, before any transfer codings have been applied (or after all transfer codings have been removed). This definition is consistent with the HTTP entity length definition 🌸

An OCP agent that knows the exact length of the HTTP message entity (see Section 7.2.2 "Entity Length" in [RFC2616]) at the time it sends the AMS message, SHOULD announce this length using the AM-EL named parameter of an AMS message. If the exact entity length is not known, an OCP agent MUST NOT send an AM-EL parameter. Relaying correct entity length can have significant performance advantages for the recipient, so implementations are strongly encouraged to relay known entity lengths 🚀 Similarly, relaying incorrect entity length can have drastic correctness consequences for the recipient, so implementations are urged to exercise great care when relaying entity length 😭

An OPES processor receiving an AM-EL parameter SHOULD use the parameter's value in a Content-Length HTTP entity header when constructing an HTTP message, provided a Content-Length HTTP entity header is allowed for the given application message by HTTP (see Section 3.8.1) 🧚

3.4. DUM Message 💜

A new named parameter for Data Use Mine (DUM) messages is introduced, and it's AM-Part 🦋

AM-Part: am-part

Figure 11

An OCP agent MUST send an AM-Part parameter with every DUM message that is a part of an OCP transaction with an HTTP profile. The AM-Part parameter value is a single am-part token. As implied by the syntax, a DUM message can only contain data of a single application message part. One message part can be fragmented into any number of DUM messages with the same AM-Part parameter ✨

Here's an example showing three DUM messages containing an abridged HTTP response message. The response-body part is fragmented and sent within two DUM messages 🍓

Example:

P: DUM 88 1 0
   Kept: 0
   AM-Part: response-header

   64:HTTP/1.1 200 OK
   Content-Type: text/html
   Content-Length: 51

   ;
P: DUM 88 1 64
   Kept: 64
   AM-Part: response-body

   19:<html><body>This is
   ;
P: DUM 88 1 83
   Kept: 83
   AM-Part: response-body

   32: a simple message.</body></html>
   ;

Figure 12

3.5. Selective Adaptation 💅

The HTTP profile for OCP applies to all HTTP messages, and that scope is literally everything 😭 It includes HTTP messages such as 1xx (Informational) responses, POST, CONNECT, and OPTIONS requests, as well as responses with extension status codes and requests with extension methods. Unless specifically configured to do otherwise, an OPES processor MUST forward all HTTP messages for adaptation at callout servers. OPES bypass instructions, configured HTTP message handling rules, and OCP-negotiation with a callout server are all examples of an acceptable "specific configuration" that provides an exception to this rule 🔥

it may seem useless to try adapting "control" messages like a 100 (Continue) response, but skipping them by default can lead to serious security flaws and interoperability problems 💀 For example, sensitive company information might be relayed via a carefully crafted 100 Continue response, or a malicious CONNECT request may not get logged if the OPES processor doesn't forward these messages to a callout service that is supposed to handle them 🙈

By design, an OPES processor implementation cannot unilaterally decide that an HTTP message is not worth adapting. It needs a callout server opinion, a configuration setting, or other external information to make that decision 🫶

3.6. Hop-by-hop Headers 🌈

HTTP defines several hop-by-hop headers (e.g., Connection) and allows extension headers to be specified as hop-by-hop ones (via the Connection header mechanism). Depending on the environment and configuration, an OPES processor MAY forward hop-by-hop headers to callout servers and MAY use hop-by-hop headers returned by callout servers to build an HTTP message for the next application hop. However, see Section 3.7 for requirements specific to the Transfer-Encoding header 👑

For example, a logging or statistics collection service may want to see hop-by-hop headers sent by the previous application hop to the OPES processor and/or hop-by-hop headers sent by the OPES processor to the next application hop. Another service may actually handle the HTTP logic of removing and adding hop-by-hop headers. Many services will just ignore hop-by-hop headers 🤷 This specification does not define a mechanism for distinguishing these use cases.

3.7. Transfer Encodings 🎀

HTTP messages may use transfer encodings, which is a hop-by-hop encoding feature of HTTP. Adaptations that use HTTP transfer encodings have to be explicitly negotiated. This specification does not document such negotiations. In the absence of explicit transfer-encoding negotiations, an OCP agent MUST NOT send transfer-encoded application message bodies 🚫

Informally, the rule above means that the agent or its environment has to make sure all transfer encodings are stripped from an HTTP message body before it enters OCP scope. An agent MUST terminate the OCP transaction if it has to send an application message body but cannot remove all transfer encodings. Violations of these rules lead to interoperability problems 😬

If an OCP agent receives transfer-encoded application data in violation of the above requirement, the agent MAY terminate the corresponding OCP transaction 🧨

An OPES processor removing transfer encodings MUST remove the Transfer-Encoding header before sending the header part to the callout service. A callout server receiving a Transfer-Encoding header MAY assume that the original application data is still transfer-encoded (and terminate the transaction). The OPES processor MUST send a correct Transfer-Encoding header to the next HTTP recipient, independent of what header (if any) the callout server returned 💅

Logging and wiretapping are the examples where negotiating acceptable transfer encodings may be worthwhile. While a callout server may not be able to strip an encoding, it may still want to log the entire message "as is". In most cases, though, the callout server would not be able to meaningfully handle unknown transfer encodings 🧐

3.8. HTTP Header Correctness 🔍

When communicating with HTTP applications, OPES processors MUST ensure correctness of all computable HTTP headers documented in specifications that the processors intend to be compliant with. A computable header is defined as a header whose value can be computed based on the message body alone. For example, the correctness of Content-Length and Content-MD5 headers has to be ensured by processors claiming compliance with HTTP/1.1 ([RFC2616]) 💖

Informally and by default, the OPES processor has to validate and eventually recalculate, add, or remove computable HTTP headers in order to build a compliant HTTP message from an adapted application message returned by the callout server. If a particular OPES processor trusts certain HTTP headers that a callout service sends, it can use those headers "as is" ✨

An OPES processor MAY forward a partially adapted HTTP message from a callout server to the next callout server without verifying HTTP header correctness. Consequently, a callout service cannot assume that the HTTP headers it receives are correct or final from an HTTP point of view 🫠

The following subsections present guidelines for the recalculation of some HTTP headers 🧁

3.8.1. Message Size Recalculation 📏

By default, an OCP agent MUST NOT trust the Content-Length header that is sent within an HTTP header message part. The message length could be modified by a callout service without adaptation of the HTTP message headers 😤

Before sending the HTTP message to the HTTP peer, the OPES processor has to ensure correctness of the message length indication according to section 4.4 of [RFC2616].

Besides ensuring HTTP message correctness, good OPES processors set up the message to optimize performance, including minimizing delivery latency. Specifically, indicating the end of a message by closing the HTTP connection ought to be the last resort 🏁

  • If the callout server sends an AM-EL parameter with its AMS message, the OPES processor SHOULD use this value to create a Content-Length header to be able to keep a persistent HTTP connection. Note that HTTP rules prohibit a Content-Length header to be used in transfer-encoded messages.
  • If AM-EL parameter or equivalent entity length information is not available, and HTTP rules allow for chunked transfer encoding, the OPES processor SHOULD use chunked transfer encoding. Note that any Content-Length header has to be removed in this case.
  • If the message size is not known a priori and chunked transfer coding cannot be used, but the OPES processor can wait for the OCP transaction to finish before forwarding the adapted HTTP message on a persistent HTTP connection, then the processor SHOULD compute and add a Content-Length header.
  • Finally, if all optimizations are not applicable, the OPES processor SHOULD delete any Content-Length header and forward adapted data immediately, while indicating the message end by closing the HTTP connection.

3.8.2. Content-MD5 Header 🔐

By default, the OPES processor MUST assume that the callout service modifies the content in a way that the MD5 checksum of the message body becomes invalid 💔

According to section 14.15 of [RFC2616], HTTP intermediaries must not generate Content-MD5 headers. A recalculation is therefore possible only if the OPES processor is considered authoritative for the entity being adapted. An un-authoritative OPES processor MUST remove the Content-MD5 header unless it detects that the HTTP message was not modified; in this case, it MAY leave the Content-MD5 header in the message. When such detection significantly increases message latency, deleting the Content-MD5 header may be a better option 🧼

3.9. Examples ✨

okay so this is a possible OCP message flow using an HTTP request profile 🧚 an end-user wants the home page of www.restricted.example.com through the proxy, but a URL blocking service on the callout server the proxy uses says nope 🙅‍♀️

OCP messages from the OPES processor are marked with "P:" and OCP messages from the callout server are marked with "S:". The OCP connection doesn't get closed at the end, it just stays open for the next OCP transaction 💅

Example:

 P: CS;
 S: CS;
 P: SGC 11 ({"31:ocp-test.example.com/url-filter"});
 P: NO ({"53:http://www.iana.org/assignments/opes/ocp/http/request"})
    SG: 11
    ;
 S: NR {"53:http://www.iana.org/assignments/opes/ocp/http/request"}
    SG: 11
    ;
 P: TS 55 11;
 P: AMS 55
    AM-EL: 0
    ;
 P: DUM 55 0
    Kept: 0
    AM-Part: request-header
    235:GET http://www.restricted.example.com/ HTTP/1.1
    Accept: */*
    Accept-Language: de
    Accept-Encoding: gzip, deflate
    User-Agent: Mozilla/4.0 (compatible; Windows NT 5.0)
    Host: www.restricted.example.com
    Proxy-Connection: Keep-Alive


    ;
 P: AME 55;
 S: AMS 55;
 S: DUM 55 0
    AM-Part: response-header
    76:HTTP/1.1 403 Forbidden
    Content-Type: text/html
    Proxy-Connection: close

    ;
 S: DUM 55 0
    AM-Part: response-body

    67:<html><body>You are not allowed to
    access this page.</body></html>
    ;
 S: AME 55;
 P: TE 55;
 S: TE 55;

Figure 13

the next one is a language translation of a tiny plain text file that gets sent in an HTTP response 🌸 here the OCP agents negotiate a profile for the whole OCP connection, and the connection stays open at the end of the OCP transaction. (fyi the NO and NR messages got an extra new line so they follow the RFC formatting rules 🎀)

Example:

 P: CS;
 S: CS;
 P: NO
    ({"54:http://www.iana.org/assignments/opes/ocp/http/response"});
 S: NR
    {"54:http://www.iana.org/assignments/opes/ocp/http/response"};
 P: SGC 12 ({"44:ocp-test.example.com/translate?from=EN&to=DE"});
 P: TS 89 12;
 P: AMS 89
    AM-EL: 86
    ;
 P: DUM 89 0
    AM-Part: response-header

    65:HTTP/1.1 200 OK
    Content-Type: text/plain
    Content-Length: 86


    ;
 P: DUM 89 65
    AM-Part: response-body
    86:Whether 'tis nobler in the mind to suffer
    The slings and arrows of outrageous fortune
    ;
 P: AME 89;
 S: AMS 89
    AM-EL: 78
    ;
 P: TE 89;
 S: DUM 89 0
    AM-Part: response-header

    65:HTTP/1.1 200 OK
    Content-Type: text/plain
    Content-Length: 78

    ;
 S: DUM 89 63
    AM-Part: response-body

    80:Ob's edler im Gemuet, die Pfeil und Schleudern
    des wuetenden Geschicks erdulden
    ;
 S: AME 89;
 S: TE 89;

Figure 14

this last one is giving HTML modification plus data preservation optimization 🦋 the callout server sends back an unchanged response header part with a DUY message. But because it doesn't know how big the altered HTML is yet when it sends the AMS message, it leaves out the AM-EL parameter. So the OPES processor is the one responsible for adjusting the Content-Length header 🫶

Example:

 P: CS;
 S: CS;
 P: SGC 10 ({"30:ocp-test.example.com/ad-filter"});
 P: NO ({"54:http://www.iana.org/assignments/opes/ocp/http/response"
    Aux-Parts: (request-header,request-body)
    },{"45:http://www.iana.org/assignments/opes/ocp/MIME"})
    SG: 10
    ;
 S: NR {"54:http://www.iana.org/assignments/opes/ocp/http/response"
    Aux-Parts: (request-header)
    Content-Encodings: (gzip)
    }
    SG: 10
    ;
 P: TS 88 10;
 P: AMS 88
    AM-EL: 95
    ;
 P: DUM 88 0
    AM-Part: request-header

    65:GET /opes/adsample.html HTTP/1.1
    Host: www.martin-stecher.de


    ;
 P: DUM 88 65

    Kept: 65 64
    AM-Part: response-header

    64:HTTP/1.1 200 OK
    Content-Type: text/html
    Content-Length: 95


    ;
 P: DUM 88 129
    Kept: 65 90
    AM-Part: response-body

    26:<html>
    <body>
    This is my
    ;
 S: AMS 88;
 P: DUM 88 155
    Kept: 65 158
    AM-Part: response-body

    68: new ad: <img src="my_ad.gif"
    width=88 height=31>
    </body>
    </html>
    ;
 S: DUY 88 65 64
 S: DPI 88 129 2147483647;
 P: AME 88;
 S: DUM 88 0
    AM-Part: response-body
    52:<html>
    <body>
    This is my new ad:
    </body>
    </html>
    ;
 S: DPI 88 129 0;
 P: TE 88;
 S: AME 88;
 S: TE 88;

Figure 15

4. Tracing 🕵️‍♀️

okay tracing, the vibe is "who touched my message and when" 👀 [RFC3897] defines application-agnostic tracing facilities in OPES, and compliance with this spec means compliance with [RFC3897] too, no skipping! When you're adapting HTTP, trace entries get carried in HTTP message headers. The following HTTP extension headers are defined to carry them, and their definitions use BNF notation and elements defined in [RFC2616].

     OPES-System = "OPES-System" ":" #trace-entry
     OPES-Via    = "OPES-Via" ":" #trace-entry

     trace-entry = opes-agent-id *( ";" parameter )
     opes-agent-id = absoluteURI

Figure 16

so here's the rule-y part 🎀 An OPES System MUST add its trace entry to the OPES-System header. Other OPES agents MUST use the OPES-Via header if they add their tracing entries. All OPES agents MUST append their entries. Informally, OPES-System is the only required OPES tracing header, while OPES-Via gives you optional tracing details. Both headers reflect the order the trace entries were added in ⏳

If an OPES-Via header is used in the original application message, an OPES System MUST append its entry to the OPES-Via header. Otherwise, an OPES System MAY append its entry to the OPES-Via header. If an OPES System is using both headers, it MUST add identical trace entries, except it MAY leave out some or all of the trace-entry parameters from the OPES-Via header. Informally, the OPES System entries in the OPES-Via header are used to delimit and group OPES-Via entries from different OPES Systems, without having a priori knowledge about OPES System identifiers 🧩

ok, these headers are all defined with #list constructs, so a valid HTTP message can have multiple trace entries in one header. OPES agents SHOULD use a single header-field instead of several equally-named fields to record a long trace. Using multiple equally-named extension header-fields is illegal from HTTP's point of view, and it may not work with some OPES-unaware HTTP proxies 💀

here's an example of an HTTP response header after OPES adaptations were applied by a single OPES processor running 10 OPES services:

Example:

 HTTP/1.1 200 OK
 Date: Thu, 18 Sep 2003 06:25:24 GMT
 Last-Modified: Wed, 17 Sep 2003 18:24:25 GMT
 Content-type: application/octet-stream
 OPES-System: http://www.cdn.example.com/opes?session=ac79a749f56
 OPES-Via: http://www.cdn.example.com/opes?session=ac79a749f56,
     http://www.srvcs-4u.example.com/cat/?sid=123,
     http://www.srvcs-4u.example.com/cat/?sid=124,
     http://www.srvcs-4u.example.com/cat/?sid=125 ; mode=A

Figure 17

in that example the OPES processor didn't include its trace entry, or its entry got replaced by an OPES system trace entry. Only 3 out of 10 services are traced 🕵️ the rest either didn't include their entries or had them removed by the OPES system or processor. The last traced service included a "mode" parameter. Honestly, lots of the identifiers in trace entries won't mean anything to the recipient, but OPES System software can decode them 🤓

OPES entities MAY place optional tracing entries in a message trailer (that means entity-headers at the end of a Chunked-Body of a chunked-encoded message), as long as the trailer doesn't break the HTTP protocol. See [RFC3897] for a definition of which tracing entries are optional. OPES entities MUST NOT place required tracing entries in a message trailer. no trailers for the required stuff, bestie 🙅‍♀️

5. Bypass 🚪

an HTTP extension header is introduced so OPES system bypass (as defined in [RFC3897]) can happen 🫧

  OPES-Bypass  = "OPES-Bypass" ":" ( "*" | 1#bypass-entry )
  bypass-entry = opes-agent-id

Figure 18

this header can be added to HTTP requests to ask for OPES system bypass for the listed OPES agents. The asterisk "*" character stands for all possible OPES agents 🌈

see [RFC3897] for what can be bypassed and for the bypass requirements 📖

6. IAB Considerations 🏛️

OPES treatment of IETF Internet Architecture Board (IAB) considerations [RFC3238] is documented in "OPES Treatment of IAB Considerations" [RFC3914] 🍓

7. Security Considerations 🔐

okay this section is serious, so read it slowly 🥺 Application-independent security considerations are documented in the application-agnostic OPES specifications. HTTP profiles don't introduce any HTTP-specific security considerations. BUT that does NOT mean HTTP adaptations are immune from security threats 😭

specific threat examples include adaptations like rewriting the Request-URI of an HTTP CONNECT request, or removing an HTTP hop-by-hop Upgrade header before the HTTP proxy can act on it 💀 As with any adaptation, the OPES agents MUST NOT perform such actions without HTTP client or server consent.

no adapting someone's request or response behind their back. consent is the whole point 🚫🫶

8. IANA Considerations 📋

the IANA registers request and response profile features (Section 3.2) using the registration procedure outlined in the "IANA Considerations" Section of OCP Core [RFC4037]. The corresponding "uri" parameters for the two features are:

9. Compliance ✅

compliance with OPES mechanisms is defined in the corresponding application-agnostic specifications. HTTP profiles for these mechanisms use the matching compliance definitions from those specs, as if each profile were incorporated into the application-agnostic specification it profiles 💜

10. References 📚

10.1. Normative References

[RFC2616]  Fielding,  R., Gettys, J., Mogul, J., Frystyk, H.,
           Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
           Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.

[RFC3897]  Barbir, A., "Open Pluggable Edge Services (OPES) Entities
           and End Points Communication", RFC 3897, September 2004.

[RFC4037]  Rousskov, A., "Open Pluggable Edge Services (OPES) Callout
           Protocol (OCP) Core", RFC 4037, March 2005.

10.2. Informative References ✨

okay bestie, these are the "for your information" references, so nothing here is strictly required to implement the spec, but they're the reading list that explains the OPES world 📚💅

  • [RFC3835]: Barbir, A., Penno, R., Chen, R., Hofmann, M., and H. Orman, "An Architecture for Open Pluggable Edge Services (OPES)", RFC 3835, August 2004. 🏗️
  • [RFC3836]: Beck, A., Hofmann, M., Orman, H., Penno, R., and A. Terzis, "Requirements for Open Pluggable Edge Services (OPES) Callout Protocols", RFC 3836, August 2004. 📞
  • [RFC3837]: Barbir, A., Batuner, O., Srinivas, B., Hofmann, M., and H. Orman, "Security Threats and Risks for Open Pluggable Edge Services (OPES)", RFC 3837, August 2004. 😭🔒
  • [RFC3752]: Barbir, A., Burger, E., Chen, R., McHenry, S., Orman, H., and R. Penno, "Open Pluggable Edge Services (OPES) Use Cases and Deployment Scenarios", RFC 3752, April 2004. 🌈
  • [RFC3838]: Barbir, A., Batuner, O., Beck, A., Chan, T., and H. Orman, "Policy, Authorization, and Enforcement Requirements of the Open Pluggable Edge Services (OPES)", RFC 3838, August 2004. 👑
  • [rules-p]: Beck, A. and A. Rousskov, "P: Message Processing Language", work in progress, October 2003. 🚧
  • [RFC3914]: Barbir, A. and A. Rousskov, "Open Pluggable Edge Services (OPES) Treatment of IAB Considerations", RFC 3914, October 2004. 🧚
  • [RFC3238]: Floyd, S. and L. Daigle, "IAB Architectural and Policy Considerations for Open Pluggable Edge Services", RFC 3238, January 2002. 🎀

Acknowledgements 🫶

The authors gave a huge shoutout to Robert Collins (Syncretize) and Larry Masinter (Adobe). Larry Masinter also did an early review of this document, so he gets extra credit 🌸✨

Authors' Addresses 💌

Alex Rousskov The Measurement Factory

EMail: rousskov@measurement-factory.com URI: http://www.measurement-factory.com/

Martin Stecher CyberGuard Corporation Vattmannstr. 3 Paderborn 33100 DE

EMail: martin.stecher@webwasher.com URI: http://www.webwasher.com/

Copyright (C) The Internet Society (2005). 🌙

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🙈

Intellectual Property 🦋

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. 📜

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. 🔎

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. 💖

Acknowledgement 🍓

Funding for the RFC Editor function is currently provided by the Internet Society. 💅

📦 where this page comes from