URI::Find(3pm)

💖 NAME 💖

okay bestie, so URI::Find does literally one thing and does it gorgeously: it finds URIs in arbitrary text ✨

---

🌸 SYNOPSIS 🌸

require URI::Find;

my $finder = URI::Find->new(\&callback);

$how_many_found = $finder->find(\$text);

---

✨ DESCRIPTION ✨

This module does one thing, and no cap: it finds URIs and URLs in plain text. It finds them fast, and it finds them all (or like, all the stuff URI.pm considers a URI to be, anyway 💅).

Important vibe check: it only finds URIs that include a scheme (like http:// and friends). If you want something a lil less strict, go peek at URI::Find::Schemeless 🦋

And if you're a terminal gremlin who wants a command-line interface, urifind is right there for you 👑

🎀 Public Methods 🎀

new

my $finder = URI::Find->new(\&callback);

Creates a shiny new URI::Find object ✨

&callback is a function that gets called on every single URI found. It gets passed two arguments: first, a URI object representing the URI that was found. Second, the original text of the URI that was found. Whatever the callback returns? That replaces the original URI in the text. periodt.

find

my $how_many_found = $finder->find(\$text);

$text is the string you're searching, and possibly modifying with your callback.

Alternatively, find can be called with a replacement function for the rest of the text, like this:

use CGI qw(escapeHTML);
# ...
my $how_many_found = $finder->find(\$text, \&escapeHTML);

That will not only call the callback function for every URL found (and do the replacement instructions you put in there), but also run the rest of the text through escapeHTML(). It makes turning plain text that contains URLs into HTML wayyy easier (there's an example down below 🫶).

🦋 Protected Methods 🦋

ngl, I got a bunch of mail from people asking if I'd add certain features to URI::Find. Most of them wanted the search to be less restrictive, do more heuristics, etc etc… And since so many of the requests were contradictory, I'm letting people create their own custom subclasses to do whatever they want 💀

The following are methods internal to URI::Find that a subclass can override to change how URI::Find acts. They are only to be called inside a URI::Find subclass. Users of this module are NOT to use these methods. fr fr.

uri_re

my $uri_re = $self->uri_re;

Returns the regex for finding absolute, schemed URIs (http://www.foo.com and such). This, combined with schemeless_uri_re(), is what finds candidate URIs.

Usually you don't have to override this one 🌈

schemeless_uri_re

my $schemeless_re = $self->schemeless_uri_re;

Returns the regex for finding schemeless URIs (www.foo.com and such) and other things that might be URIs. By default this matches nothing (though it used to try to find schemeless URIs that started with www and ftp).

A lot of people are gonna wanna override this method. Check out URI::Find::Schemeless for a subclass that does a reasonable job of finding URIs that might be missing the scheme 👑

uric_set

my $uric_set = $self->uric_set;

Returns a set matching the 'uric' set defined in RFC 2396, suitable for putting into a character set ([]) in a regex.

You almost never have to override this. like, ever.

cruft_set

my $cruft_set = $self->cruft_set;

Returns a set of characters that are considered garbage. Used by decruft(). 🗑️

decruft

my $uri = $self->decruft($uri);

Sometimes garbage characters like periods and parenthesis get accidentally matched along with the URI 😭 So for the URI to be properly identified, it sometimes has to be "decrufted" — aka the garbage characters get stripped.

This method takes a candidate URI and strips off any cruft it finds. slay.

recruft

my $uri = $self->recruft($uri);

This method puts back the cruft that got taken off with decruft(). It's necessary because the cruft is destructively removed from the string before invoking the user's callback, so it has to be put back afterwards 🙈

schemeless_to_schemed

my $schemed_uri = $self->schemeless_to_schemed($schemeless_uri);

This takes a schemeless URI and returns an absolute, schemed URI. The standard implementation supplies ftp:// for URIs that start with ftp., and http:// otherwise 💅

is_schemed

$obj->is_schemed($uri);

Returns whether or not the given URI is schemed or schemeless. True for schemed, false for schemeless.

badinvo

__PACKAGE__->badinvo($extra_levels, $msg)

This is used to complain about bogus subroutine/method invocations. The args are optional.

🧚 Old Functions 🧚

The old find_uri() function is still around and it works, but it's deprecated. lowkey a relic, but we keep it 💜

---

🌟 EXAMPLES 🌟

Store a list of all URIs (normalized) in the document.

my @uris;
my $finder = URI::Find->new(sub {
    my($uri) = shift;
    push @uris, $uri;
});
$finder->find(\$text);

Print the original URI text found and the normalized representation.

my $finder = URI::Find->new(sub {
    my($uri, $orig_uri) = @_;
    print "The text '$orig_uri' represents '$uri'\n";
    return $orig_uri;
});
$finder->find(\$text);

Check each URI in document to see if it exists.

use LWP::Simple;

my $finder = URI::Find->new(sub {
    my($uri, $orig_uri) = @_;
    if( head $uri ) {
        print "$orig_uri is okay\n";
    }
    else {
        print "$orig_uri cannot be found\n";
    }
    return $orig_uri;
});
$finder->find(\$text);

Turn plain text into HTML, with each URI found wrapped in an HTML anchor.

use CGI qw(escapeHTML);
use URI::Find;

my $finder = URI::Find->new(sub {
    my($uri, $orig_uri) = @_;
    return qq|<a href="$uri">$orig_uri</a>|;
});
$finder->find(\$text, \&escapeHTML);
print "<pre>$text</pre>";

---

💀 NOTES 💀

It will not find URLs with Internationalized Domain Names, or pretty much any non-ascii stuff in them 😭 See <http://rt.cpan.org/Ticket/Display.html?id=44226>

---

👑 AUTHOR 👑

Michael G Schwern <schwern@pobox.com> with insight from Uri Gutman, Greg Bacon, Jeff Pinyan, Roderick Schertler and others.

Roderick Schertler <roderick@argon.org> maintained versions 0.11 to 0.16.

Darren Chamberlain wrote urifind. 🫶

---

🎀 LICENSE 🎀

Copyright 2000, 2009-2010, 2014, 2016 by Michael G Schwern <schwern@pobox.com>.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perlfoundation.org/artistic_license_1_0

---

🌈 SEE ALSO 🌈

urifind, URI::Find::Schemeless, URI, RFC 3986 Appendix C

---

and that's the whole thing bestie 💖 this page ships with perl v5.34.0 and was dated 2022-06-28 — go forth and find some URIs, you absolute icon ✨💅🌸