Net::Async::HTTP(3pm)

📦
version 0.50 · 7 versions
, in debian trixie, ubuntu resolute, ubuntu stonking

💖 Net::Async::HTTP(3pm) 💖

okay bestie, welcome to the manual page for Net::Async::HTTP from the User Contributed Perl Documentation. It's the async HTTP user agent that runs on IO::Async, and yes, it eats. ✨

✨ NAME ✨

Net::Async::HTTP — use HTTP with IO::Async. periodt.

🌈 SYNOPSIS 🌈

use Future::AsyncAwait;

use IO::Async::Loop;
use Net::Async::HTTP;
use URI;

my $loop = IO::Async::Loop->new();

my $http = Net::Async::HTTP->new();

$loop->add( $http );

my $response = await $http->do_request(
   uri => URI->new( "http://www.cpan.org/" ),
);

print "Front page of http://www.cpan.org/ is:\n";
print $response->as_string;

🦋 DESCRIPTION 🦋

This object class is literally an asynchronous HTTP user agent. It sends requests to servers and hands you back Future instances that yield responses when they show up. It supports multiple concurrent connections to servers, and it lets multiple requests be pipelined to any one connection. Normally, you only need one of these objects per program to support any number of requests. main character energy, no cap.

As well as using futures, the module also supports a callback-based interface. pick your vibe.

This module optionally supports SSL connections if IO::Async::SSL is installed. If it is, SSL can be requested either by passing a URI with the https scheme, or by passing a true value as the SSL parameter. secure girlie behaviour. 🔐

🎀 Connection Pooling 🎀

There are three ways this object manages connections to HTTP server hosts, and they're controlled by the value of max_connections_per_host. That value controls when new connections are established to servers, compared to waiting for existing connections to be free as new requests are made to them.

They are:

  • max_connections_per_host = 1 — this is the default setting. In this mode, there will be one connection per host on which there are active or pending requests. If new requests are made while an existing one is outstanding, they will be queued to wait for it.

If pipelining is active on the connection (because both the pipeline option is true and the connection is known to be an HTTP/1.1 server), then requests will be pipelined into the connection awaiting their response. If not, they will be queued awaiting a response to the previous before sending the next.

  • max_connections_per_host > 1 — in this mode, there can be more than one connection per host. If a new request is made, it will try to re-use idle connections if there are any, or if they are all busy it will create a new connection to the host, up to the configured limit.
  • max_connections_per_host = 0 — in this mode, there is no upper limit to the number of connections per host. Every new request will try to reuse an idle connection, or else create a new one if all the existing ones are busy.

These modes all apply per hostname / server port pair; they do not affect the behaviour of connections made to differing hostnames, or differing ports on the same hostname. iykyk.

🫶 PARAMETERS 🫶

The following named parameters may be passed to new or configure:

user_agent => STRING

A string to set in the User-Agent HTTP header. If not supplied, one will be constructed that declares Net::Async::HTTP and the version number.

headers => ARRAY or HASH

Since version 0.45.

A set of extra headers to apply to every outgoing request. May be specified either as an even-sized array containing key/value pairs, or a hash.

Individual header values may be added or changed without replacing the entire set by using the configure method and passing a key called +headers:

$http->configure( +headers => { One_More => "Key" } );

max_redirects => INT

Optional. How many levels of redirection to follow. If not supplied, will default to 3. Give 0 to disable redirection entirely.

max_in_flight => INT

Optional. The maximum number of in-flight requests to allow per host when pipelining is enabled and supported on that host. If more requests are made over this limit they will be queued internally by the object and not sent to the server until responses are received. If not supplied, will default to 4. Give 0 to disable the limit entirely.

max_connections_per_host => INT

Optional. Controls the maximum number of connections per hostname/server port pair, before requests will be queued awaiting one to be free. Give 0 to disable the limit entirely. See also the Connection Pooling section documented above.

Currently, if not supplied it will default to 1. However, it has been found in practice that most programs will raise this limit to something higher, perhaps 3 or 4. Therefore, a future version of this module may set a higher value.

To test if your application will handle this correctly, you can set a different default by setting an environment variable:

$ NET_ASYNC_HTTP_MAXCONNS=3 perl ...

timeout => NUM

Optional. How long in seconds to wait before giving up on a request. If not supplied then no default will be applied, and no timeout will take place.

stall_timeout => NUM

Optional. How long in seconds to wait after each write or read of data on a socket, before giving up on a request. This may be more useful than timeout on large-file operations, as it will not time out provided that regular progress is still being made.

proxy_host => STRING

proxy_port => INT

Since version 0.10.

proxy_path => PATH

Since version 0.49.

Optional. Default values to apply to each request method.

Optional. A reference to a HTTP::Cookies object. Will be used to set cookies in requests and store them from responses. cookie monster vibes. 🍪

pipeline => BOOL

Optional. If false, disables HTTP/1.1-style request pipelining.

close_after_request => BOOL

Since version 0.45.

Optional. If true, will set the Connection: close header on outgoing requests and disable pipelining, thus making every request use a new connection.

family => INT

local_host => STRING

local_port => INT

local_addrs => ARRAY

local_addr => HASH or ARRAY

Optional. Parameters to pass on to the connect method used to connect sockets to HTTP servers. Sets the socket family and local socket address to bind() to. For more detail, see the documentation in IO::Async::Connector.

fail_on_error => BOOL

Optional. Affects the behaviour of response handling when a 4xx or 5xx response code is received. When false, these responses will be processed as other responses and yielded as the result of the future, or passed to the on_response callback. When true, such an error response causes the future to fail, or the on_error callback to be invoked.

The HTTP response and request objects will be passed as well as the code and message, and the failure name will be http.

( $code_message, "http", $response, $request ) = $f->failure

$on_error->( "$code $message", $response, $request )

read_len => INT

write_len => INT

Optional. Used to set the reading and writing buffer lengths on the underlying IO::Async::Stream objects that represent connections to the server. If it's not defined, a default of 64 KiB will be used.

ip_tos => INT or STRING

Optional. Used to set the IP_TOS socket option on client sockets. If given, should either be a IPTOS_* constant, or one of the string names lowdelay, throughput, reliability or mincost. If undefined or left absent, no option will be set.

decode_content => BOOL

Optional. If true, incoming responses that have a recognised Content-Encoding are handled by the module, and decompressed content is passed to the body handling callback or returned in the HTTP::Response. See CONTENT DECODING below for details of which encoding types are recognised. When this option is enabled, outgoing requests also have the Accept-Encoding header added to them if it does not already exist.

Currently the default is false, because this behaviour is new, but it may default to true in a later version. Applications which care which behaviour applies should set this to a defined value to ensure it doesn't change.

SSL_*

Additionally, any parameters whose names start with SSL_ will be stored and passed on requests to perform SSL requests. This simplifies configuration of common SSL parameters.

require_SSL => BOOL

Optional. If true, then any attempt to make a request that does not use SSL (either by calling request, or as a result of a redirection) will immediately fail.

SOCKS_*

Since version 0.42.

Additionally, any parameters whose names start with SOCKS_ will be stored and used by Net::Async::SOCKS to establish connections via a configured proxy.

✨ METHODS ✨

Okay bestie, this is the part you actually came for. 💅 Heads up: all the methods down below are documented as an "await" expression, and they hand you back Future instances. That's the vibe.

When one of these returns a Future, any HTTP-level error comes back as the Future failure name http. If the error is tied to a specific response, that response gets included too. And the original request rides along as well. Periodt:

$f->fail( $message, "http", $response, $request )

🌸 do_request

$response = await $http->do_request( %args );

Send an HTTP request to a server, and get back a Future that'll eventually yield the response. 🦋 Depending on what args you pass, the request might be represented by an HTTP::Request object, or a URI object. Pick your fighter.

These are the named arguments you use for "HTTP::Request"s:

  • request => HTTP::Request — a reference to an "HTTP::Request" object.
  • host => STRING — the hostname of the server you're connecting to.
  • port => INT or STRING — optional, bestie. Port number or service of the server to connect to. If you don't define it, it'll default to "http" or "https", depending on whether SSL is being used.
  • family => INT or STRING — optional. Restricts the socket family used for connecting. Leave it undefined and it'll fall back to the globally-configured value on the object. The value can be a "PF_*" constant directly, or the lowercase name of one, like "inet".
  • SSL => BOOL — optional. Set it true and boom, an SSL connection gets used.

Now these are the named arguments for "URI" requests:

  • uri => URI or STRING — a reference to a "URI" object, or just a plain string with the request URI. If the scheme is "https" then an SSL connection will be used. No cap.
  • method => STRING — optional. The HTTP method name. If it's missing, "GET" is used.
  • content => STRING or ARRAY ref — optional. The body content to use for "PUT" or "POST" requests.
    • If it's a plain scalar, it gets used directly, and you must also supply a "content_type" field to describe it.
    • If it's an ARRAY ref and the request method is "POST", it gets form encoded. It should hold an even-sized list of field names and values. More deets in "POST" in HTTP::Request::Common.
  • content_type => STRING — the type of non-form data content. Don't skip this one.
  • user => STRING / pass => STRING — optional. Give both and the HTTP Basic Authorization header gets sent with these details.
  • headers => ARRAY|HASH — optional. If you provide it, it holds extra HTTP headers to set on the constructed request object. If you pass it as an ARRAY reference, it should contain an even-sized list of name/value pairs.
  • proxy_host => STRING / proxy_port => INT — Since version 0.10. Optional. Overrides the hostname or port number that the URI implies.
  • proxy_path => PATH — Since version 0.49. Optional. Sets a UNIX socket path to use as a proxy. To actually make use of this, also set the "family" argument to "unix". iykyk. 🧚

And for either request type, it also takes these:

  • request_body => STRING | CODE | Future — optional. Lets the request body content be generated by a future or a callback instead of being provided as part of the "request" object. This can be a plain string, a "CODE" reference to a generator function, or a future.

Since this gets passed to the underlying IO::Async::Stream "write" method, the usual semantics apply here. If you pass a "CODE" reference, it gets called repeatedly whenever it's safe to write, and that code should return "undef" to signal it's done. If you pass a "Future", it's expected to eventually yield the body value.

Same as with the "content" parameter, the "content_type" field should be specified explicitly in the request header, and so should the content length (typically via the HTTP::Request "content_length" method). Peep examples/PUT.pl too. 👀

  • expect_continue => BOOL — optional. If true, it sets the "Expect" request header to the value "100-continue" and it will not send the "request_body" parameter until a "100 Continue" response comes back from the server. If an error response arrives, then the "request_body" code — if present — will not be invoked.
  • on_ready => CODE — optional. A callback invoked once a socket connection is established with the HTTP server, but before the request is actually sent over it. Your client code can use this to inspect the socket, or do whatever else to it. This code is expected to return a "Future"; only once that future has completed will the request cycle continue. If it fails, that failure gets propagated to the caller.
$f = $on_ready->( $connection );
  • on_redirect => CODE — optional. A callback invoked if a redirect response is received, before the new location gets fetched. It's passed the response and the new URL.
$on_redirect->( $response, $location );
  • on_body_write => CODE — optional. A callback invoked after each successful "syswrite" of the body content. Perfect for an upload progress indicator or something like that. It's passed the total number of bytes of body content written so far (so that's excluding bytes consumed in the header, bestie).
$on_body_write->( $written );
  • max_redirects => INT — optional. How many levels of redirection to follow. If you don't supply it, it defaults to the value given in the constructor.
  • timeout => NUM / stall_timeout => NUM — optional. Overrides the object's configured timeout values for this one request. Leave them unspecified and the configured defaults get used.

On a timeout, the returned future fails with either "timeout" or "stall_timeout" as the operation name.

( $message, "timeout" ) = $f->failure;

🙈 do_request (void)

$http->do_request( %args );

When you're not returning a future, these extra arguments get used as callbacks instead:

  • on_response => CODE — a callback invoked when a response to this request has been received. It gets passed an HTTP::Response object containing the response the server sent.
$on_response->( $response );
  • on_header => CODE — an alternative to "on_response". A callback invoked when the header of a response has been received. It's expected to return a "CODE" reference for handling chunks of body content. That "CODE" reference gets invoked with no arguments once the end of the request has been reached, and whatever it returns becomes the result of the returned "Future", if there is one.
$on_body_chunk = $on_header->( $header );

   $on_body_chunk->( $data );
   $response = $on_body_chunk->();
  • on_error => CODE — a callback invoked if an error happens while trying to send the request or get the response. It gets passed an error message.
$on_error->( $message );

If this gets invoked because of a received "4xx" or "5xx" error code in an HTTP response, it'll be invoked with the response and request objects too.

$on_error->( $message, $response, $request );

💖 GET, HEAD, PUT, ...

$response = await $http->GET( $uri, %args );

$response = await $http->HEAD( $uri, %args );

$response = await $http->PUT( $uri, $content, %args );

$response = await $http->POST( $uri, $content, %args );

Since version 0.36.

$response = await $http->PATCH( $uri, $content, %args );

Since version 0.48.

$response = await $http->DELETE( $uri, %args );

Since version 0.49.

These are the convenient little wrappers for doing "GET", "HEAD", "PUT", "POST", "PATCH" or "DELETE" requests with a "URI" object and barely any other arguments, returning a "Future". So easy. ✨

Don't forget: "POST" with non-form data (which is what you get when you pass a plain scalar instead of an "ARRAY" reference of form data name/value pairs) needs a "content_type" key in %args. It will simply not work otherwise, bestie.

🎀 SUBCLASS METHODS 🎀

These next ones are meant as points for subclasses to override, so you can add extra functionality of your own. Main character energy only. 👑

prepare_request

$http->prepare_request( $request );

Called just before the "HTTP::Request" object is sent to the server.

process_response

$http->process_response( $response );

Called after a non-redirect "HTTP::Response" has been received from a server. The originating request will be set in the object.

🌈 CONTENT DECODING 🌈

If the required decompression modules are installed and available, compressed content can be decoded. 🔥 If the received "Content-Encoding" is recognised and the required module is available, the content gets transparently decoded, and the decoded content is what gets returned in the resulting response object — or passed to the data chunk handler. In that case, the original "Content-Encoding" header gets deleted from the response, and its value becomes available instead as "X-Original-Content-Encoding". Sneaky but iconic.

These are the content encoding types recognised by these modules:

  • gzip (q=0.7) and deflate (q=0.5)

Recognised if Compress::Raw::Zlib version 2.057 or newer is installed.

  • bzip2 (q=0.8)

Recognised if Compress::Bzip2 version 2.10 or newer is installed.

Other content encoding types can be registered by calling this method:

register_decoder

Net::Async::HTTP->register_decoder( $name, $q, $make_decoder )

Registers an encoding type called $name, at the quality value $q. To decode this encoding type, $make_decoder gets invoked with no parameters, and is expected to return a CODE reference that performs one instance of decoding.

$decoder = $make_decoder->()

This decoder gets invoked on string buffers to decode them until the end of stream is reached, at which point it gets invoked with no arguments. 💅

$content = $decoder->( $encoded_content )
$content = $decoder->() # EOS

✨ EXAMPLES ✨

ok bestie this is the part where it all comes together and you get to see the vibe in action 💖

🌈 Concurrent GET 🌈

the Future-returning GET method? yeah it makes it so easy to await a whole bunch of URLs at once, and you do it with the Future::Utils fmap_void utility. it's giving multitasking queen 👑

use Future::AsyncAwait;
use Future::Utils qw( fmap_void );

my @URLs = ( ... );

my $http = Net::Async::HTTP->new( ... );
$loop->add( $http );

my $future = fmap_void {
   my ( $url ) = @_;
   $http->GET( $url )
        ->on_done( sub {
           my $response = shift;
           say "$url succeeded: ", $response->code;
           say "  Content-Type:", $response->content_type;
        } )
        ->on_fail( sub {
           my $failure = shift;
           say "$url failed: $failure";
        } );
} foreach => \@URLs,
  concurrent => 5;

await $future;

like look at that. five at a time, no cap, and every single one reports back whether it ate or flopped 🍓

👀 SEE ALSO 👀

(the whole lore, iykyk 📜)

💸 SPONSORS 💸

some of this code, or the bugfixes to it, were literally paid for by these icons. real ones fr fr 🫶

👑 AUTHOR 👑

Paul Evans <leonerd@leonerd.org.uk>

the main character who wrote all this. absolute legend, no notes 💅

and that's a wrap bestie!! 🎀 this page came from perl v5.40.0, dated 2024-10-27, and it's Net::Async::HTTP(3pm) in the User Contributed Perl Documentation. go make some async requests and slay ✨🦋

📦 where this page comes from