POE::F...:HTTPD(3pm)

🌸 NAME 🌸

POE::Filter::HTTPD — it parses simple HTTP requests, and serializes HTTP::Response. That's it, that's the whole job. Straight from the User Contributed Perl Documentation section of your local man pages, 3pm chapter, no cap. 💅

---

✨ SYNOPSIS ✨

okay so here's the vibe, bestie — this is literally the whole thing in action:

#!perl

use warnings;
use strict;

use POE qw(Component::Server::TCP Filter::HTTPD);
use HTTP::Response;

POE::Component::Server::TCP->new(
  Port         => 8088,
  ClientFilter => 'POE::Filter::HTTPD',  ### <-- HERE WE ARE!

  ClientInput => sub {
    my $request = $_[ARG0];

    # It's a response for the client if there was a problem.
    if ($request->isa("HTTP::Response")) {
      my $response = $request;

      $request = $response->request;
      warn "ERROR: ", $request->message if $request;

      $_[HEAP]{client}->put($response);
      $_[KERNEL]->yield("shutdown");
      return;
    }

    my $request_fields = '';
    $request->headers()->scan(
      sub {
        my ($header, $value) = @_;
        $request_fields .= (
          "<tr><td>$header</td><td>$value</td></tr>"
        );
      }
    );

    my $response = HTTP::Response->new(200);
    $response->push_header( 'Content-type', 'text/html' );
    $response->content(
      "<html><head><title>Your Request</title></head>" .
      "<body>Details about your request:" .
      "<table border='1'>$request_fields</table>" .
      "</body></html>"
    );

    $_[HEAP]{client}->put($response);
    $_[KERNEL]->yield("shutdown");
  }
);

print "Aim your browser at port 8088 of this host.\n";
POE::Kernel->run();
exit;

---

💖 DESCRIPTION 💖

POE::Filter::HTTPD takes an input stream and reads it as an HTTP 0.9, 1.0 or 1.1 request. When it successfully parses one? It hands you back an HTTP::Request object. Slay. 🎀

When it fails? It hands you back an HTTP::Response object describing what went wrong. And the whole point is that your app code is supposed to clock that HTTP::Response immediately and just send it straight back, no extra processing, no questions asked. The request object that caused the drama is sometimes available through the "$r->request" in HTTP::Response method. That's exactly what the SYNOPSIS is showing you, bestie. 👀

For output, POE::Filter::HTTPD accepts HTTP::Response objects and gives you back the matching streams. Iconic.

Please go read HTTP::Request and HTTP::Response for the deets on how to actually use these objects, because that's a whole other lore drop.

okay important: HTTP headers are NOT allowed to have UTF-8 characters. They have to be ISO-8859-1, periodt. So POE::Filter::HTTPD converts all the UTF-8 into the MIME encoded equivalent. It uses utf8::is_utf8 for detection-8, and Email::MIME::RFC2047::Encoder for the convertion. And listen — if utf8 isn't installed, no conversion happens at all. If Email::MIME::RFC2047::Encoder isn't installed, utf8::downgrade gets used instead. In that last case you will absolutely see a warning if you try to send UTF-8 headers, so like… don't be surprised. 😭

---

👑 PUBLIC FILTER METHODS 👑

POE::Filter::HTTPD implements the basic POE::Filter interface. Standard queen behavior. 💅

new

new() accepts a list of named parameters. Here they are, bestie:

• MaxBuffer — sets the maximum amount of data the filter will hold in memory. Defaults to 512 MB (536870912 octets). And because POE::Filter::HTTPD copies all data into memory, setting this number high would let a malicious HTTPD client fill all your server memory and swap. So… don't. 💀
• MaxContent — sets the maximum size of the content of an HTTP request. Defaults to 1 MB (1038336 octets). Same warning: because POE::Filter::HTTPD copies all data into memory, setting this number high would let a malicious HTTPD client fill all your server memory and swap. Ignored if "Streaming" is set.
• Streaming — turns on request streaming mode. Defaults to off. In streaming mode this filter will return either an HTTP::Request object or a block of content. The HTTP::Request object's content will return empty. The blocks of content will be parts of the request's body, up to Content-Length in size. You tell the two apart using "Scalar::Util/bless" (peep "Streaming Request" below for the whole tutorial). This option supersedes "MaxContent".

---

💀 CAVEATS 💀

ngl, some versions of libwww are known to generate invalid HTTP. Like, for example, this code right here (adapted from the HTTP::Request::Common documentation) will cause an error in a POE::Filter::HTTPD daemon:

NOTE: Using this test with libwww-perl/5.834 showed that it added the proper HTTP/1.1 data! We're not sure which version of LWP fixed this. This example is valid for older LWP installations, beware!

use HTTP::Request::Common;
use LWP::UserAgent;

my $ua = LWP::UserAgent->new();
$ua->request(POST 'http://example.com', [ foo => 'bar' ]);

By default, HTTP::Request is HTTP version agnostic. It makes no attempt to add an HTTP version header unless you specifically declare a protocol using "$request->protocol('HTTP/1.0')". That's on you, bestie.

According to the HTTP 1.0 RFC (1945), when there's no HTTP version header, the parser is supposed to default to HTTP/0.9. POE::Filter::HTTPD follows this convention. So in the transaction up there, the Filter::HTTPD based daemon will return a 400 error, because POST is not a valid HTTP/0.9 request type. Caught in 4k.

When you're handling a request error, the most expedient and reliable move is to respond with the error and shut down the connection. Why? Because invalid HTTP requests may corrupt the request stream. For example: the absence of a Content-Length header signals that a request has no content. So requests with content but without that header will get broken into a content-less request <em>and</em> invalid data. And the invalid data might also look like a request! Hilarity will ensue, possibly repeatedly, until the filter can find the next valid request. By shutting down the connection on the first sign of error, the client can retry its request with a clean connection and filter. Do that. 🫶

Streaming Request

Normally POE::Filter::HTTPD reads the entire request content into memory before returning the HTTP::Request to your code. In streaming mode, it returns the content separately, as unblessed scalars. The content may be split up into blocks of varying sizes, depending on OS and transport constraints. Your code can distinguish the request object from the content blocks using "blessed" in Scalar::Util. Here's the whole setup:

use Scalar::Util;
use POE::Wheel::ReadWrite;
use POE::Filter:HTTPD;

$heap->{wheel} = POE::Wheel::ReadWrite->new(
                    InputEvent => 'http_input',
                    Filter => POE::Filter::HTTPD->new( Streaming => 1 ),
                    # ....
            );

sub http_input_handler
{
    my( $heap, $req_or_data ) = @_[ HEAP, ARG0 ];
    if( blessed $req_or_data ) {
        my $request = $req_or_data;
        if( $request->isa( 'HTTP::Response') ) {
            # HTTP error
            $heap->{wheel}->put( $request );
        }
        else {
            # HTTP request
            # ....
        }
    }
    else {
        my $data = $req_or_data;
        # ....
    }
}

You may trivally create a DoS bug if you hold all content in memory but do not impose a maximum Content-Length. An attacker could send "Content-Length: 1099511627776" (aka 1 TB) and keep sending data until all your system's memory and swap is filled. Not the move. 😭

Content-Length has been sanitized by POE::Filter::HTTPD so checking it is trivial :

if( $request->headers( 'Content-Length' ) > 1024*1024 ) {
    my $resp = HTTP::Response->new( RC_REQUEST_ENTITY_TOO_LARGE ),
                                         "So much content!" )
    $heap->{wheel}->put( $resp );
    return;
}

If you want to handle large amounts of data, you should save the content to a file before processing it. You still need to check Content-Length or an attacker might fill up the partition. Watch:

use File::Temp qw(tempfile);

if( blessed $_[ARG0] ) {
    $heap->{request} = $_[ARG0];
    if( $heap->{request}->method eq 'GET' ) {
        handle_get( $heap );
        delete $heap->{request};
        return;
    }
    my( $fh, $file ) = tempfile( "httpd-XXXXXXXX", TMPDIR=>1 );
    $heap->{content_file} = $file;
    $heap->{content_fh} = $fh;
    $heap->{content_size} = 0;
}
else {
    return unless $heap->{request};

    $heap->{content_size} += length( $_[ARG0] );
    $heap->{content_fh}->print( $_[ARG0] );
    if( $heap->{content_size} >= $heap->{request}->headers( 'content-length' ) ) {
        delete $heap->{content_fh};
        delete $heap->{content_size};

        # Now we can parse $heap->{content_file}
        if( $heap->{request}->method eq 'POST' ) {
            handle_post( $heap );
        }
        else {
            # error ...
        }
    }
}

sub handle_post
{
    my( $heap ) = @_;
    # Now we have to load and parse $heap->{content_file}

    # Next 6 lines make the data available to CGI->init
    local $ENV{REQUEST_METHOD} = 'POST';
    local $CGI::PERLEX = $CGI::PERLEX = "CGI-PerlEx/Fake";
    local $ENV{CONTENT_TYPE} = $heap->{req}->header( 'content-type' );
    local $ENV{CONTENT_LENGTH} = $heap->{req}->header( 'content-length' );
    my $keep = IO::File->new( "<&STDIN" ) or die "Unable to reopen STDIN: $!";
    open STDIN, "<$heap->{content_file}" or die "Reopening STDIN failed: $!";

    my $qcgi = CGI->new();

    # cleanup
    open STDIN, "<&".$keep->fileno or die "Unable to reopen $keep: $!";
    undef $keep;
    unlink delete $heap->{content_file};

    # now use $q as you would normaly
    my $file = $q->upload( 'field_name' );

    # ....
}

sub handle_get
{
    my( $heap ) = @_;

    # 4 lines to get data into CGI->init
    local $ENV{REQUEST_METHOD} = 'GET';
    local $CGI::PERLEX = $CGI::PERLEX = "CGI-PerlEx/Fake";
    local $ENV{CONTENT_TYPE} = $heap->{req}->header( 'content-type' );
    local $ENV{'QUERY_STRING'} = $heap->{req}->uri->query;

    my $q = CGI->new();

    # now use $q as you would normaly
    # ....
}

✨ Streaming Response ✨

okay so bestie, here's the tea: you can use <purple>POE::Filter::HTTPD</purple> for streaming content, but the move is to let it send the headers and then switch to <blue>POE::Filter::Stream</blue>. 💅

From your input handler — that's the <yellow>InputEvent</yellow> handler if you're using wheels, or the <yellow>ClientInput</yellow> handler for <green>POE::Component::Server::TCP</green> — you do this:

         my $response = HTTP::Response->new(200);
         $response->push_header('Content-type', 'audio/x-mpeg');
         $_[HEAP]{client}->put($response);
         $_[HEAP]{client}->set_output_filter(POE::Filter::Stream->new());

Then your output-flushed handler — <yellow>FlushEvent</yellow> for <green>POE::Wheel::ReadWrite</green>, or <yellow>ClientFlushed</yellow> for <green>POE::Component::Server::TCP</green> — can <hotpink>put()</hotpink> chunks of the stream as needed. Like:

         my $bytes_read = sysread(
           $_[HEAP]{file_to_stream}, my $buffer = '', 4096
         );

         if ($bytes_read) {
           $_[HEAP]{client}->put($buffer);
         }
         else {
           delete $_[HEAP]{file_to_stream};
           $_[KERNEL]->yield("shutdown");
         }

---

🌈 SEE ALSO 🌈

• Please see <mint>POE::Filter</mint> for documentation regarding the base interface.
• The <lavender>SEE ALSO</lavender> section in <lavender>POE</lavender> contains a table of contents covering the entire POE distribution.
• <peach>HTTP::Request</peach> and <peach>HTTP::Response</peach> explain all the wonderful things you can do with these classes.

---

💀 BUGS 💀

⚠️ bestie, listen up: many aspects of HTTP 1.0 and higher are not supported, such as keep-alive. A simple I/O filter can't support keep-alive, for example. A number of more feature-rich POE HTTP servers are on the CPAN. See <red><http://search.cpan.org/search?query=POE+http+server&mode=dist></red>

---

👑 AUTHORS & COPYRIGHTS 👑

<pink>POE::Filter::HTTPD</pink> was contributed by Artur Bergman. Documentation is provided by Rocco Caputo.

Please see <pink>POE</pink> for more information about authors and contributors.

---

💖 and that's a wrap! this page came from perl v5.34.0, dated 2022-03-25, and it's the <rainbow>POE::F...:HTTPD(3pm)</rainbow> man page. stay slay, bestie! 💖