RFC 6202

Known Issues and Best Practices for the Use of Long Polling and Streaming in Bidirectional HTTP

📅 published April 2011, from ietf

RFC 6202: Known Issues and Best Practices for the Use of Long Polling and Streaming in Bidirectional HTTP 🎀

Okay bestie, here's the whole vibe. It's an Informational RFC from the IETF, published April 2011, with ISSN 2070-1721. The authors are S. Loreto (Ericsson), P. Saint-Andre (Cisco), S. Salsano (University of Rome "Tor Vergata"), and G. Wilkins (Webtide). It doesn't obsolete or update anything, since the title block lists no Obsoletes or Updates entries. 💅

Abstract 🦋

So on today's Internet, HTTP gets used (some would say abused) for asynchronous, "server-initiated" communication from a server to a client, plus the usual client-to-server stuff. ✨ This doc breaks down the known issues and best practices for these "bidirectional HTTP" apps, with a focus on the two most popular mechanisms: HTTP long polling and HTTP streaming. 🌸

Status of This Memo 📜

This document is not an Internet Standards Track specification. It's published purely for informational purposes. 💜

It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved document is a candidate for any level of Internet Standard, so check out Section 2 of RFC 5741 for the details. 🧚

Info about the current status of this document, any errata, and how to give feedback can be found at http://www.rfc-editor.org/info/rfc6202. 🔥

Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. 🫶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License. ⭐

Table of Contents 🗂️

1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
2.  HTTP Long Polling  . . . . . . . . . . . . . . . . . . . . . .  4
  2.1.  Definition . . . . . . . . . . . . . . . . . . . . . . . .  4
  2.2.  HTTP Long Polling Issues . . . . . . . . . . . . . . . . .  5
3.  HTTP Streaming . . . . . . . . . . . . . . . . . . . . . . . .  7
  3.1.  Definition . . . . . . . . . . . . . . . . . . . . . . . .  7
  3.2.  HTTP Streaming Issues  . . . . . . . . . . . . . . . . . .  8
4.  Overview of Technologies . . . . . . . . . . . . . . . . . . . 10
  4.1.  Bayeux . . . . . . . . . . . . . . . . . . . . . . . . . . 10
  4.2.  BOSH . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
  4.3.  Server-Sent Events . . . . . . . . . . . . . . . . . . . . 13
5.  HTTP Best Practices  . . . . . . . . . . . . . . . . . . . . . 13
  5.1.  Limits to the Maximum Number of Connections  . . . . . . . 13
  5.2.  Pipelined Connections  . . . . . . . . . . . . . . . . . . 14
  5.3.  Proxies  . . . . . . . . . . . . . . . . . . . . . . . . . 14
  5.4.  HTTP Responses . . . . . . . . . . . . . . . . . . . . . . 15
  5.5.  Timeouts . . . . . . . . . . . . . . . . . . . . . . . . . 15
  5.6.  Impact on Intermediary Entities  . . . . . . . . . . . . . 16
6.  Security Considerations  . . . . . . . . . . . . . . . . . . . 16
7.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 17
  7.1.  Normative References . . . . . . . . . . . . . . . . . . . 17
  7.2.  Informative References . . . . . . . . . . . . . . . . . . 17
8.  Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 18

1. Introduction 🌸

Okay so, HTTP ([RFC2616]) is a request/response protocol. It defines three kinds of entities: clients, proxies, and servers. A client opens connections to a server so it can send HTTP requests. A server accepts connections from clients to service those requests by sending back responses. Proxies are the middlemen that can get involved in carrying requests and responses from the client to the server and back again. 🦋

In the standard HTTP model, a server cannot start a connection with a client, and it can't send a response nobody asked for. 💀 So the server can't push asynchronous events to clients on its own. To get events ASAP, the client has to keep polling the server for new content. The catch is that constant polling can eat up a ton of bandwidth, because every poll forces a request/response round trip even when there's nothing new. It's also inefficient, since the app gets less responsive: data just sits in a queue until the server gets the client's next poll request. 🙈

To fix this, a bunch of server-push programming mechanisms have popped up in recent years. They're often grouped under the label "Comet" ([COMET]) and let a web server send updates to clients without waiting for a poll request. These mechanisms can deliver updates more promptly and skip the latency that apps suffer from constantly opening and closing connections just to poll for data. ✨

The two most common server-push mechanisms are HTTP long polling and HTTP streaming:

  • HTTP Long Polling: The server tries to "hold open" each HTTP request, meaning it doesn't reply right away. It only responds when there are events to deliver. That way there's always a pending request the server can answer to push events as they happen, which keeps message delivery latency low. 🍓
  • HTTP Streaming: The server keeps a request open indefinitely. It never ends the request or closes the connection, even after it has pushed data to the client. 🌈

Other technologies for bidirectional HTTP are possible, but they usually need changes to HTTP itself (for example, defining new HTTP methods). This document only looks at bidirectional HTTP technologies that work within the current scope of HTTP as defined in [RFC2616] (HTTP 1.1) and [RFC1945] (HTTP 1.0). 👑

The authors admit that both HTTP long polling and HTTP streaming stretch the original meaning of HTTP, and that HTTP was never designed for bidirectional communication. 🙈 This document neither encourages nor discourages these mechanisms, and it doesn't take a stance on whether they're the right answer for bidirectional communication between clients and servers. What it does do is call out the technical issues with these mechanisms and suggest best practices for deploying them. 💅

Here's the roadmap. Section 2 analyzes HTTP long polling. Section 3 analyzes HTTP streaming. Section 4 gives an overview of the specific technologies that use server push. Section 5 lists best practices for bidirectional HTTP using existing technologies. 🗺️

2. HTTP Long Polling 🎀

2.1. Definition 💖

With traditional "short polling," the client sends regular requests to the server, and each one tries to "pull" any available events or data. If nothing is available, the server returns an empty response, and the client waits a bit before sending another poll request. How often it polls depends on how much latency the client can tolerate when getting updates. The downside is that the resources used (server processing and network) depend heavily on how much latency is acceptable for delivering updates from server to client. If the acceptable latency is low (say, on the order of seconds), the polling frequency can become an unacceptable burden on the server, the network, or both. 😭

"Long polling" is the glow-up. It tries to minimize both the latency of server-to-client message delivery and the use of processing and network resources. The server gets these efficiencies by only responding to a request once a particular event, status, or timeout has occurred. Once the server sends a long poll response, the client typically sends a new long poll request right away. So at any given moment the server is holding open a long poll request, and it replies when new information is available for the client. That's why the server can asynchronously "initiate" communication. 🔥

The basic life cycle of an application using HTTP long polling goes like this:

  1. The client makes an initial request and then waits for a response.
  2. The server defers its response until an update is available or until a particular status or timeout has occurred.
  3. When an update is available, the server sends a complete response to the client.
  4. The client typically sends a new long poll request, either right after receiving a response or after a pause to allow an acceptable latency period.

HTTP long polling can be used with either persistent or non-persistent HTTP connections. Persistent connections avoid the extra overhead of setting up a new TCP/IP connection ([TCP]) for every long poll request, which is very much a main character move. 🦋

2.2. HTTP Long Polling Issues ✨

Okay so long polling comes with a few issues you need to know about, bestie 🫶

  • Header Overhead: 🧾 With HTTP long polling, every long poll request and every long poll response is a complete HTTP message, so each one carries a full set of HTTP headers in its framing. For small, infrequent messages, those headers can be a huge chunk of everything that gets sent. If the network MTU (Maximum Transmission Unit) lets all the information, HTTP header included, fit inside a single IP packet, this usually isn't a big deal for networking entities. But the amount of transferred data can end up way bigger than the real payload HTTP is carrying, and that can really matter when volume-based charging is in place 💸
  • Maximal Latency: ⏱️ After a long poll response goes out to a client, the server has to wait for the next long poll request before it can send another message. So the average latency of long polling is close to one network transit, but the maximal latency is over three network transits (long poll response, next long poll request, long poll response). Also, HTTP rides on TCP/IP, so packet loss and retransmission can happen, which means the maximal latency for any TCP/IP protocol will be more than three network transits (lost packet, next packet, negative ack, retransmit) 😭 When HTTP pipelining (see Section 5.2) is available, the latency from the server waiting on a new request can be avoided ✨
  • Connection Establishment: 🔌 People often say short polling and long polling keep opening TCP/IP connections and then closing them. But both polling mechanisms work well with persistent HTTP connections that can be reused for lots of poll requests. The pause between a long poll response and the next long poll request is short, so idle connections don't get closed in the middle 🙌
  • Allocated Resources: 🧠 Operating systems allocate resources to TCP/IP connections and to the HTTP requests outstanding on those connections. HTTP long polling needs both a TCP/IP connection and an HTTP request held open for each client, so you have to think about both when you size a long polling application. The resources used per TCP/IP connection are usually minimal and can scale reasonably well. The resources allocated to HTTP requests, though, can be significant, and the total number of outstanding requests can be limited on some gateways, proxies, and servers 🚧
  • Graceful Degradation: 🌊 A long polling client or server under load naturally degrades gracefully in performance, and the cost is message latency. If load makes a client or server run slowly, the events to be pushed to the client will queue up. They're waiting either for the client to send a long poll request, or for the server to free up CPU cycles so it can process a long poll request that's being held. When several messages are queued for a client, they might get delivered together in a batch within a single long poll response. That can significantly reduce the per-message overhead and ease the workload of the client or server for the given message load 💪
  • Timeouts: ⏳ Long poll requests have to stay pending, or "hanging," until the server has something to send to the client. The timeout issues for these pending requests are covered in Section 5.5.
  • Caching: 🗂️ Caching mechanisms run by intermediate entities can interfere with long poll requests. This issue is covered in Section 5.6.

3. HTTP Streaming 🌊

3.1. Definition

Okay, HTTP streaming is giving "never let go" energy 💖 It keeps a request open indefinitely. It never terminates the request or closes the connection, even after the server pushes data to the client. Because the client and the server don't have to keep opening and closing the connection, this mechanism significantly reduces network latency ⚡

The basic life cycle of an application using HTTP streaming goes like this:

  1. The client makes an initial request and then waits for a response.
  2. The server defers the response to a poll request until an update is available, or until a particular status or timeout has occurred.
  3. Whenever an update is available, the server sends it back to the client as a part of the response.
  4. The data sent by the server does not terminate the request or the connection. The server returns to step 3.

The HTTP streaming mechanism is built on the server's ability to send several pieces of information in the same response, without terminating the request or the connection. HTTP/1.1 servers and HTTP/1.0 servers can both achieve this 🎀

An HTTP response content length can be defined using three options, and yes they're all different vibes 💅

  • Content-Length header: This indicates the size of the entity body in the message, in bytes.
  • Transfer-Encoding header: The 'chunked' value in this header indicates the message will break into chunks of known size if needed.
  • End of File (EOF): This is actually the default approach for HTTP/1.0, where connections are not persistent. Clients don't need to know the size of the body they're reading. Instead, they expect to keep reading the body until the server closes the connection. HTTP/1.1 defaults to persistent connections, but it's still possible to use EOF by setting the 'Connection:close' header in either the request or the response. That tells everyone the connection is not to be considered 'persistent' once the current request/response is complete. If the client includes the 'Connection: close' header field in its request, that will also prevent pipelining 🙃

The main issue with EOF is that it's hard to tell the difference between a connection that was terminated by a fault and one that was correctly terminated 😬

An HTTP/1.0 server can only use EOF as a streaming mechanism. An HTTP/1.1 server, on the other hand, has both EOF and "chunked transfer" available to it ✨

The "chunked transfer" mechanism is the one HTTP/1.1 servers typically use for streaming. It works by including the header Transfer-Encoding: chunked at the beginning of the response. That lets the server send the following parts of the response in different "chunks" over the same connection. Each chunk starts with the hexadecimal expression of the length of its data, followed by CR/LF, and the end of the response is marked by a chunk of size 0 💫

        HTTP/1.1 200 OK
        Content-Type: text/plain
        Transfer-Encoding: chunked

        25
        This is the data in the first chunk

        1C
        and this is the second one

        0

                Figure 1: Transfer-Encoding response

To get the same result, an HTTP/1.0 server omits the Content-Length header in the response. That way it can still send the subsequent parts of the response on the same connection. (In this case, the different parts of the response aren't explicitly separated by the HTTP protocol, and the end of the response happens when the connection gets closed 🥹)

3.2. HTTP Streaming Issues

Here are the issues that come with HTTP streaming, and they're a lot 😤

  • Network Intermediaries: 🚦 HTTP allows intermediaries (proxies, transparent proxies, gateways, etc.) to be involved in sending a response from the server to the client. There's no requirement for an intermediary to forward a partial response right away, and it's totally legal for an intermediary to buffer the entire response before sending any data to the client (for example, caching transparent proxies). HTTP streaming won't work with those intermediaries 🙈
  • Maximal Latency: 📈 In theory, on a perfect network, an HTTP streaming protocol's average and maximal latency is one network transit. In practice, though, the maximal latency is higher because of network and browser limitations. The browser techniques used to end HTTP streaming connections are often tied to JavaScript and/or DOM (Document Object Model) elements that grow in size for every message received. So, to keep client memory usage from growing forever, an HTTP streaming implementation occasionally has to end the streaming response and send a request to start a new streaming response (which is basically the same as a long poll). That means the maximal latency is at least three network transits. And because HTTP rides on TCP/IP, packet loss and retransmission can happen, so the maximal latency for any TCP/IP protocol will be more than three network transits (lost packet, next packet, negative ack, retransmit) 💀
  • Client Buffering: 🧺 Existing HTTP specifications don't require a client library to make the data from a partial HTTP response available to the client application. For example, if each response chunk contains a statement of JavaScript, there's no requirement for the browser to run that JavaScript before the entire response is received. In practice, though, most browsers do execute JavaScript received in partial responses, although some need a buffer overflow to trigger execution. In most implementations, blocks of white space can be sent to achieve that buffer overflow 🫠
  • Framing Techniques: 🧩 With HTTP streaming, several application messages can be sent within a single HTTP response. Separating the response stream into application messages has to happen at the application level, not at the HTTP level. In particular, you can't use the HTTP chunks as application message delimiters, because intermediate proxies might "re-chunk" the message stream (for example, by combining different chunks into a longer one) 🙅‍♀️ This issue doesn't affect HTTP long polling, which gives you a canonical framing technique: each application message can be sent in a different HTTP response 🎉

4. Overview of Technologies 🧚

This section gives you a quick tour of the existing technologies that implement HTTP-based server-push mechanisms to deliver messages from the server to the client asynchronously. Let's go 💜

4.1. Bayeux 🦋

okay so the Bayeux protocol [BAYEUX] was built in 2006-2007 by the Dojo Foundation 💖. It can use both the HTTP long polling and HTTP streaming mechanisms ✨

To get bidirectional communication, a Bayeux client uses two HTTP connections to a Bayeux server, so server-to-client and client-to-server messages can both happen asynchronously. bestie, that's basically two lanes of traffic 🛣️

The Bayeux specification requires that implementations control pipelining of HTTP requests, so requests don't get pipelined inappropriately (e.g., a client-to-server message pipelined behind a long poll request). lowkey the whole point is that a message shouldn't get stuck waiting behind a long poll 🙈

In practice, for JavaScript clients, that control over pipelining is not possible in current browsers 💀. So JavaScript implementations of Bayeux try to meet the requirement by limiting themselves to a maximum of two outstanding HTTP requests at any one time. That way the browser connection limits won't be applied, and the requests won't be queued or pipelined. It's broadly effective, but this mechanism can be disrupted if non-Bayeux JavaScript clients simultaneously issue requests to the same host 😬

Bayeux connections are negotiated between client and server with handshake messages, which let the connection type, authentication method, and other parameters get agreed on. During the handshake phase, the client and the server also reveal to each other their acceptable bidirectional techniques, and the client picks one from the intersection of those sets 🤝

For non-browser or same-domain Bayeux, clients use HTTP POST requests to the server for both the long poll request and the request to send messages to the server. The Bayeux protocol packets are sent as the body of the HTTP messages using the "application/json" Internet media type [RFC4627].

For browsers operating in cross-domain mode, Bayeux tries to use Cross-Origin Resource Sharing [CORS] checking if the browser and server support it, so normal HTTP POST requests can be used. If that fails, Bayeux clients use the "JSONP" mechanism as described in [JSONP]. In that last case, client-to-server messages are sent as encoded JSON on the URL query parameters, and server-to-client messages are sent as a JavaScript program that wraps the message JSON with a JavaScript function call to the already loaded Bayeux implementation 🌈

4.2. BOSH 🎀

BOSH, which stands for Bidirectional-streams Over Synchronous HTTP [BOSH], was developed by the XMPP Standards Foundation in 2003-2004. Its purpose is to emulate normal TCP connections over HTTP. (TCP is the standard connection mechanism used in the Extensible Messaging and Presence Protocol as described in [RFC6120].) BOSH uses the HTTP long polling mechanism by letting the server (called a "BOSH connection manager") hold off on its response to a request until it actually has data to send to the client from the application server itself (typically an XMPP server). As soon as the client gets a response from the connection manager, it sends another request, so the connection manager is (almost) always holding a request it can use to "push" data to the client 🧚

In some situations, the client needs to send data to the server while it's waiting for data to be pushed from the connection manager. To keep data from being pipelined behind the long poll request that's on hold, the client can send its outbound data in a second HTTP request over a second TCP connection. BOSH forces the server to respond to the request it's been holding on the first connection as soon as it gets a new request from the client, even if it has no data to send to the client. It does this so the client can send more data immediately, if necessary -- even when the client can't pipeline the requests -- while also respecting the two-connection limit discussed in Section 5.1 🫶

The number of long poll request-response pairs is negotiated during the first request sent from the client to the connection manager. Typically, BOSH clients and connection managers negotiate two pairs, although it's possible to use only one pair or more than two.

The roles of the two request-response pairs typically switch whenever the client sends data to the connection manager. So when the client issues a new request, the connection manager immediately answers the blocked request on the other TCP connection, freeing it up. In a scenario where only the client sends data, the even requests go over one connection and the odd ones go over the other 🔄

BOSH can work reliably both when network conditions force every HTTP request to be made over a different TCP connection and when it's possible to use HTTP/1.1 and then rely on two persistent TCP connections.

If the connection manager has no data to send to the client for an agreed amount of time (also negotiated during the first request), then it responds to the held request with no data, and that response immediately triggers a fresh client request. It does this so that if a network connection breaks, both parties realize it within a reasonable amount of time 👀

Moreover, BOSH defines the negotiation of an "inactivity period" value that specifies the longest allowable inactivity period (in seconds). That lets the client make sure the periods with no requests pending are never too long ⏰

BOSH allows data to be pushed immediately when HTTP pipelining is available. However, if HTTP pipelining is not available and one of the endpoints has just pushed some data, BOSH will usually need to wait for a network round-trip time until the server is able to push data to the client again.

BOSH uses standard HTTP POST request and response bodies to encode all information.

BOSH normally uses HTTP pipelining over a persistent HTTP/1.1 connection. However, a client can deliver its POST requests in any way permitted by HTTP 1.0 or HTTP 1.1. (Although the use of HTTP POST with pipelining is discouraged in RFC 2616, BOSH uses various methods, such as request identifiers, to make sure this usage doesn't lead to indeterminate results if the transport connection is terminated prematurely.)

BOSH clients and connection managers are not allowed to use Chunked Transfer Coding, since intermediaries might buffer each partial HTTP request or response and only forward the full request or response once it's available.

BOSH allows the use of the Accept-Encoding and Content-Encoding headers in the request and in the response, respectively, and then compresses the response body accordingly.

Each BOSH session can share the HTTP connection(s) it uses with other HTTP traffic, including other BOSH sessions and HTTP requests and responses completely unrelated to the BOSH protocol (e.g., Web page downloads).

4.3. Server-Sent Events 🍓

The W3C Server-Sent Events specification [WD-eventsource] defines an API that lets servers push data to Web pages over HTTP in the form of DOM events.

The data is encoded as "text/event-stream" content and pushed using an HTTP streaming mechanism. The specification suggests disabling HTTP chunking for serving event streams unless the rate of messages is high enough to avoid the possible negative effects of this technique, as described in Section 3.2.

However, it's not clear if there are significant benefits to using EOF rather than chunking with regards to intermediaries, unless they support only HTTP/1.0. 🤷

5. HTTP Best Practices 💅

5.1. Limits to the Maximum Number of Connections 🚧

HTTP [RFC2616], Section 8.1.4, recommends that a single user client not maintain more than two connections to any server or proxy, in order to prevent the server from being overloaded and to avoid unexpected side effects in congested networks. Until recently, this limit was implemented by most commonly deployed browsers, which made connections a scarce resource that needed to be shared within the browser. Note that the available JavaScript APIs in the browsers hide the connections, and the security model inhibits the sharing of any resource between frames. The new HTTP specification [HTTPBIS] removes the two-connection limitation, only encouraging clients to be conservative when opening multiple connections. In fact, recent browsers have increased this limit to 6 or 8 connections; however, it is still not possible to discover the local limit, and usage of multiple frames and tabs still places 8 connections within easy reach 😭

Web applications need to limit the number of long poll requests initiated, ideally to a single long poll that is shared between frames, tabs, or windows of the same browser. However, the security constraints of the browsers make such sharing difficult.

A best practice for a server is to use cookies [COOKIE] to detect multiple long poll requests from the same browser, and to avoid deferring both requests, since this might cause connection starvation and/or pipeline issues.

5.2. Pipelined Connections 🎶

HTTP [RFC2616] permits optional request pipelining over persistent connections. Multiple requests can be enqueued before the responses arrive.

In the case of HTTP long polling, the use of HTTP pipelining can reduce latency when multiple messages need to be sent by a server to a client in a short period of time. With HTTP pipelining, the server can receive and enqueue a set of HTTP requests. Therefore, the server does not need to receive a new HTTP request from the client after it has sent a message to the client within an HTTP response. In principle, HTTP pipelining can be applied to HTTP GET and HTTP POST requests, but using HTTP POST requests is more critical. In fact, the use of HTTP POST with pipelining is discouraged in RFC 2616 and needs to be handled with special care.

There is an issue regarding the inability to control pipelining. Normal requests can be pipelined behind a long poll, and are thus delayed until the long poll completes.

Mechanisms for bidirectional HTTP that want to exploit HTTP pipelining need to verify that HTTP pipelining is available (e.g., supported by the client, the intermediaries, and the server). If it's not available, they need to fall back to solutions without HTTP pipelining.

5.3. Proxies

okay so proxies are lowkey the main character here 🎀 Most of them work great with HTTP long polling, because a complete HTTP response gets sent either when an event happens or when a timeout hits 💖 Proxies are advised to pass that response straight back to the user agent right away, and then the user agent acts on it immediately ✨

HTTP streaming is more chaotic 😭 It uses partial responses and sends some JavaScript inside an HTTP/1.1 chunk (see Section 3). Two things can go wrong. First, it counts on proxies to forward each chunk, even though nothing actually requires them to, and some caching proxies just don't. Second, it counts on user agents to run each chunk of JavaScript as it arrives, which nothing requires them to do either 💀

A "reverse proxy" is basically a proxy that acts like the real server to any client or client-side proxy, but then forwards the request on to the actual server, which is usually sitting behind another layer of firewalls 🧚 Any HTTP short polling or HTTP long polling solution works fine with this, and so do most HTTP streaming solutions. The main downside is performance, since most proxies aren't built to hold a lot of open connections 🙈

Reverse proxies can get into trouble when they try to share connections to the servers between multiple clients 😬 For example, Apache with mod_jk shares a small set of connections (often 8 or 16) between all clients. If long polls get sent over those shared connections, the proxy can run out of connections, and then other requests (long poll or normal) can get held up. So Comet mechanisms currently need to avoid any connection sharing, either in the browser or in any intermediary, because HTTP assumes each request will complete as fast as possible 🏃‍♀️

One of the big reasons HTTP long polling and HTTP streaming get a bad rep with servers and proxies is that they use a synchronous programming model for handling requests. The resources allocated to each request stay tied up for the whole duration of that request. Asynchronous proxies and servers can handle long polls using slightly more resources than normal HTTP traffic. Unfortunately, some synchronous proxies do exist (e.g., Apache mod_jk), and many HTTP application servers also use a blocking model for request handling (e.g., the Java servlet 2.5 specification) 🥹

5.4. HTTP Responses

Per [RFC2616], the server answers a request it has successfully received with a 200 OK response, but only once a particular event, status, or timeout has occurred 🎯 The body of that 200 OK response contains the actual event, status, or timeout that happened. This "best practice" is literally just standard HTTP, no cap 💅

5.5. Timeouts

The HTTP long polling mechanism lets the server respond to a request only when a particular event, status, or timeout has occurred ⏳ To keep latency in server-to-client message delivery low, and to keep the processing and network resources needed as small as possible, the long poll request timeout ought to be set to a high value.

BUT the timeout value has to be picked carefully, bestie 😭 Problems can show up if it's set too high. For example, the client might get a 408 Request Timeout answer from the server, or a 504 Gateway Timeout answer from a proxy. The default timeout value in a browser is 300 seconds, but most network setups include proxies and servers whose timeouts aren't that long 🙃

Several experiments have worked with timeouts as high as 120 seconds, but generally 30 seconds is the safer value 🧸 So vendors of network equipment that want to be compatible with the HTTP long polling mechanism are advised to implement a timeout substantially greater than 30 seconds, where "substantially" means several times more than the medium network transit time 🌸

5.6. Impact on Intermediary Entities

There's no way for an end client or host to tell HTTP intermediaries that long polling is in use 🫣 So long poll requests are completely transparent to intermediary entities and get handled as normal requests. This can affect intermediary entities that do operations which aren't useful for long polling. However, any capabilities that might get in the way of bidirectional flow (e.g., caching) can be controlled with standard headers or cookies 🍪

As a best practice, caching is always intentionally suppressed in a long poll request or response. In other words, the Cache-Control header is set to no-cache 🙌

6. Security Considerations 🔐

Okay, this part is serious, so listen up 🫶 This document is meant to describe how HTTP is currently used to enable asynchronous or server-initiated communication. It does not propose any change to the HTTP protocol or to the expected behavior of HTTP entities. So this document does not introduce new security concerns into existing HTTP infrastructure. The considerations below only refer to solutions that are already implemented and deployed.

One security concern with cross-domain HTTP long polling is that the mechanism is often implemented by executing the JavaScript returned from the long poll request. If the server is prone to injection attacks, then it could be far easier to trick a browser into executing the code [CORS]. That's a big 💀 moment.

Another security concern is that the number of open connections a server has to maintain for HTTP long polling and HTTP streaming could more easily lead to denial-of-service (DoS) attacks [RFC4732] 🚨

7. References

7.1. Normative References

[RFC1945]         Berners-Lee, T., Fielding, R., and H. Nielsen,
                  "Hypertext Transfer Protocol -- HTTP/1.0",
                  RFC 1945, May 1996.

[RFC2616]         Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
                  Masinter, L., Leach, P., and T. Berners-Lee,
                  "Hypertext Transfer Protocol -- HTTP/1.1",
                  RFC 2616, June 1999.

[RFC4732]         Handley, M., Rescorla, E., and IAB, "Internet
                  Denial-of-Service Considerations", RFC 4732,
                  December 2006.

7.2. Informative References

[BAYEUX]          Russell, A., Wilkins, G., Davis, D., and M.
                  Nesbitt, "Bayeux Protocol -- Bayeux 1.0.0", 2007,
                  <http://svn.cometd.com/trunk/bayeux/bayeux.html>.

[BOSH]            Paterson, I., Smith, D., and P. Saint-Andre,
                  "Bidirectional-streams Over Synchronous HTTP
                  (BOSH)", XSF XEP 0124, February 2007.

[COMET]           Russell, A., "Comet: Low Latency Data for the
                  Browser", March 2006, <http://infrequently.org/
                  2006/03/comet-low-latency-data-for-the-browser/ >.

[COOKIE]          Barth, A., "HTTP State Management Mechanism", Work
                  in Progress, March 2011.

[CORS]            van Kesteren, A., "Cross-Origin Resource Sharing",
                  W3C Working Draft WD-cors-20100727, latest version
                  available at <http://www.w3.org/TR/cors/>,
                  July 2010,
                  <http://www.w3.org/TR/2010/WD-cors-20100727/>.

[HTTPBIS]         Fielding, R., Ed., Gettys, J., Mogul, J., Nielsen,
                  H., Masinter, L., Leach, P., Berners-Lee, T.,
                  Lafon, Y., Ed., and J. Reschke, Ed., "HTTP/1.1,
                  part 1: URIs, Connections, and Message Parsing",
                  Work in Progress, March 2011.

[JSONP]           Wikipedia, "JSON with padding",
                  <http://en.wikipedia.org/wiki/JSONP#JSONP>.

[RFC4627]         Crockford, D., "The application/json Media Type for
                  JavaScript Object Notation (JSON)", RFC 4627,
                  July 2006.

[RFC6120]         Saint-Andre, P., "Extensible Messaging and Presence
                  Protocol (XMPP): Core", RFC 6120, March 2011.

[TCP]             Postel, J., "Transmission Control Protocol", STD 7,
                  RFC 793, September 1981.

[WD-eventsource]  Hickson, I., "Server-Sent Events", W3C Working
                  Draft WD-eventsource-20091222, latest version
                  available at <http://www.w3.org/TR/eventsource/>,
                  December 2009, <http://www.w3.org/TR/2009/
                  WD-eventsource-20091222/>.

8. Acknowledgments 🌟

Huge thanks to Joe Hildebrand, Julien Laganier, Jack Moffitt, Subramanian Moonesamy, Mark Nottingham, Julian Reschke, Martin Thomson, and Martin Tyler for their feedback 🫶💖

Authors' Addresses ✍️

Salvatore Loreto
Ericsson
Hirsalantie 11
Jorvas  02420
Finland

EMail: salvatore.loreto@ericsson.com


Peter Saint-Andre
Cisco
1899 Wyknoop Street, Suite 600
Denver, CO  80202
USA

Phone: +1-303-308-3282
EMail: psaintan@cisco.com


Stefano Salsano
University of Rome "Tor Vergata"
Via del Politecnico, 1
Rome  00133
Italy

EMail: stefano.salsano@uniroma2.it


Greg Wilkins
Webtide

EMail: gregw@webtide.com
📦 where this page comes from