Web::Query(3pm)

📦

version 1.01 · 2 versions

1.01 · debian trixie, ubuntu noble, ubuntu resolute, ubuntu stonking (latest)
0.39 · debian bookworm, ubuntu jammy

, in debian trixie, ubuntu noble, ubuntu resolute, ubuntu stonking

💖 Web::Query 💖

okay bestie, this is the manual page for Web::Query (the Web::Query(3pm) one) — perl's little scraping framework that's giving serious jQuery energy, and honestly? iconic. let's get into it 🫶

✨ NAME ✨

Web::Query — yet another scraping library like jQuery

✨ VERSION ✨

version 1.01

✨ SYNOPSIS ✨

use Web::Query;

wq('http://www.w3.org/TR/html401/')
    ->find('div.head dt')
    ->each(sub {
        my $i = shift;
        printf("%d %s\n", $i+1, $_->text);
    });

✨ DESCRIPTION ✨

Web::Query is yet another scraping framework, and it comes with a jQuery-like interface. yes. we know. we've heard of it.

okay so real talk: yes, I know about Ingy's pQuery. but bestie... it's just alpha quality. it doesn't work. 😭 so Web::Query is built on top of the CPAN modules HTML::TreeBuilder::XPath, LWP::UserAgent, and HTML::Selector::XPath. built different, fr fr. 👑

so here's the tea: this module uses HTML::Selector::XPath and it only supports the CSS 3 selectors that that module supports. Web::Query does not support jQuery's extended queries (yet? 👀). and if you pass a selector as a scalar ref, it'll be taken as a straight XPath expression instead.

$wq( '<div><p>hello</p><p>there</p></div>' )->find( 'p' );       # css selector
$wq( '<div><p>hello</p><p>there</p></div>' )->find( \'/div/p' ); # xpath selector

🚨 THIS LIBRARY IS UNDER DEVELOPMENT. ANY API MAY CHANGE WITHOUT NOTICE. 🚨 like, don't get attached to any of the method names, bestie. it's giving situationship. 💀

✨ FUNCTIONS ✨

wq($stuff) — this is just a lil shortcut for Web::Query->new($stuff). and guess what, it's exported by default 🎀 so you can just start calling it immediately like it's your bestie.

✨ METHODS ✨

🌸 CONSTRUCTORS 🌸

my $q = Web::Query->new($stuff, \%options ) — makes you a brand new Web::Query instance. you can build one from a URL (http, https, file scheme), HTML in a string, a URL in a string, a URI object, undef, and either one HTML::Element object or an array ref of them.

# all valid creators
$q = Web::Query->new( 'http://techblog.babyl.ca' );
$q = Web::Query->new( '<p>foo</p>' );
$q = Web::Query->new( undef );

if you hand it a $stuff it doesn't recognise, this method throws an exception 💀. and if you gave it a URL and the response wasn't successful, it returns undef.

right now the only two valid options are indent, which gets used as the indentation string when the object is printed, and no_space_compacting, which stops whitespace characters from being compacted inside text blocks. those are the only two, bestie, don't go inventing more.

my $q = Web::Query->new_from_element($element: HTML::Element) — makes a fresh Web::Query instance out of an HTML::Element instance. 🦋
"my $q = Web::Query->new_from_html($html: Str)" — makes a fresh Web::Query instance out of HTML. ✨
my $q = Web::Query->new_from_url($url: Str) — makes a fresh Web::Query instance out of a URL.

if the response is not success (and by that I mean it doesn't match /^20[0-9]$/), this method returns undef. 🙈

you can grab the last response result with $Web::Query::RESPONSE.

here's the best practical code, and it's genuinely the move:

my $url = 'http://example.com/';
my $q = Web::Query->new_from_url($url)
    or die "Cannot get a resource from $url: " . Web::Query->last_response()->status_line;

my $q = Web::Query->new_from_file($file_name: Str) — makes a fresh Web::Query instance out of a file name. 📁

🌈 TRAVERSING 🌈

`add`

returns a new object augmented with the new element(s).

add($html) — an HTML fragment to add to the set of matched elements.
add(@elements) — one or more @elements to add to the set of matched elements.

any @elements that are already part of the set do not get added a second time. no duplicates in this house. 🚫

my $group = $wq->find('#foo');         # collection has 1 element
$group = $group->add( '#bar', $wq );   # 2 elements
$group->add( '#foo', $wq );            # still 2 elements

add($wq) — an existing Web::Query object to add to the set of matched elements.
add($selector, $context) — $selector is a string representing a selector expression, used to find extra elements to add to the set of matched elements.

$context is the point in the document where the selector should start matching. 📍

`contents`

gets the immediate children of each element in the set of matched elements, and yes that includes text and comment nodes too. everything's invited. 🎀

`each`

visits each node. $i is a counter value, 0 origin. $elem is the iteration item. and $_ gets localized to $elem. 💅

$q->each(sub { my ($i, $elem) = @_; ... })

`end`

goes back to the before context, just like jQuery does. ⏪

`filter`

reduces the elements down to only the ones that pass the function's test.

$q->filter(sub { my ($i, $elem) = @_; ... })

`find`

gets the descendants of each element in the current set of matched elements, filtered by a selector.

my $q2 = $q->find($selector); # $selector is a CSS3 selector.

NOTE if you want to match the element itself, use "filter". 🫶

⚠️ INCOMPATIBLE CHANGE — from v0.14 to v0.19 (inclusive), find() also matched the element itself, which is not jQuery compatible. you can get that same result by using filter(), add() and find() together: perl my $wq = wq('<div class="foo"><p class="foo">bar</p></div>'); # needed because we don't have a global document like jQuery does print $wq->filter('.foo')->add($wq->find('.foo'))->as_html; # <div class="foo"><p class="foo">bar</p></div><p class="foo">bar</p>

`first`

returns the first matching element. 🥇

this method constructs a new Web::Query object from that first matching element.

`last`

returns the last matching element.

this method constructs a new Web::Query object from that last matching element.

`match($selector)`

returns a boolean telling you whether the elements match $selector.

in scalar context, it only returns the boolean for the first element.

for the reverse of not(), see filter(). 🔁

`not($selector)`

returns all the elements that do not match $selector.

# $do_for_love will be every thing, except #that
my $do_for_love = $wq->find('thing')->not('#that');

`and_back`

adds the previous set of elements to the current one. bringing the old crew along. 🫂

# get the h1 plus everything until the next h1
$wq->find('h1')->next_until('h1')->and_back;

`map`

creates a new array with the results of calling a provided function on every element.

$q->map(sub { my ($i, $elem) = @_; ... })

`parent`

gets the parent of each element in the current set of matched elements.

`prev`

gets the previous node of each element in the current set of matched elements.

my $prev = $q->prev;

`next`

gets the next node of each element in the current set of matched elements.

my $next = $q->next;

`next_until( $selector )`

gets all subsequent siblings, up to (but not including) the next node matched by $selector.

💅 MANIPULATION 💅

`add_class`

adds the specified class(es) to each of the set of matched elements. ✨

# add class 'foo' to <p> elements
wq('<div><p>foo</p><p>bar</p></div>')->find('p')->add_class('foo');

`toggle_class( @classes )`

toggles the given class or classes on each of the elements. so like, if the element had the class, it gets removed, and if it hadn't, it gets added. 🔄

classes are toggled once, no matter how many times they show up in the argument list. so no double-toggling shenanigans.

$q->toggle_class( 'foo', 'foo', 'bar' );

# equivalent to

$q->toggle_class('foo')->toggle_class('bar');

# and not

$q->toggle_class('foo')->toggle_class('foo')->toggle_class('bar');

`after`

inserts content, specified by the parameter, after each element in the set of matched elements.

wq('<div><p>foo</p></div>')->find('p')
                           ->after('<b>bar</b>')
                           ->end
                           ->as_html; # <div><p>foo</p><b>bar</b></div>

the content can be anything accepted by "new". 🫶

`append`

inserts content, specified by the parameter, to the end of each element in the set of matched elements.

wq('<div></div>')->append('<p>foo</p>')->as_html; # <div><p>foo</p></div>

the content can be anything accepted by "new".

`as_html`

returns the string representations of either the first or all elements, depending on whether you called it in list or scalar context. 👀

if you give it the argument "join", the string representations of the elements get concatenated together using the given string.

wq( '<div><p>foo</p><p>bar</p></div>' )
    ->find('p')
    ->as_html( join => '!' );
# <p>foo</p>!<p>bar</p>

`" attr "`

get/set attribute values.

in getter mode, it returns either the values of the attribute for all elements of the set, or just the first one, depending on the calling context.

my @values = $q->attr('style');      # style of all elements
my $first_value = $q->attr('style'); # style of first element

in setter mode, it sets the attribute value for all elements, and hands the original object right back to you so chaining stays easy. 💅

$q->attr( 'alt' => 'a picture' )->find( ... );

# can pass more than 1 element too
$q->attr( alt => 'a picture', src => 'file:///...' );

the value you pass for an attribute can be a code ref. in that case, the code gets called with $_ set to the current attribute value. and if the code modifies $_, the attribute gets updated with the new value. ✨

$q->attr( alt => sub { $_ ||= 'A picture' } );

`" id "`

get/set the elements's id attribute.

in getter mode, it behaves just like attr().

in setter mode, it behaves like attr() too, but with these exceptions:

if the attribute value is a scalar, it only gets assigned to the first element of the set (because ids are supposed to be unique, bestie, that's the whole point 👑), and the returned object will only contain that first element.

my $first_element = $q->id('the_one');

it's possible to set the ids of all the elements by passing a sub to id(). the sub gets given the same arguments as for each(), and its return value is taken to be the new id of the elements.

$q->id( sub { my $i = shift;  'foo_' . $i } );

`" name "`

get/set the elements's 'name' attribute.

my $name = $q->name;  # equivalent to $q->attr( 'name' );

$q->name( 'foo' );    # equivalent to $q->attr( name => 'foo' );

`" data "`

get/set the elements's 'data-*name*' attributes. 🍓

my $data = $q->data('foo');  # equivalent to $q->attr( 'data-foo' );

$q->data( 'foo' => 'bar' );  # equivalent to $q->attr( 'data-foo' => 'bar' );

`tagname`

get/set the tag name of elements.

my $name = $q->tagname;

$q->tagname($new_name);

`before`

inserts content, specified by the parameter, before each element in the set of matched elements.

wq('<div><p>foo</p></div>')->find('p')
                           ->before('<b>bar</b>')
                           ->end
                           ->as_html; # <div><b>bar</b><p>foo</p></div>

the content can be anything accepted by "new".

`clone`

creates a deep copy of the set of matched elements. twins, basically. 👯

`detach`

removes the set of matched elements from the DOM. bye bestie 👋

`has_class`

Okay so bestie, this one? It determines whether any of the matched elements are assigned the given class. 💅 Like, does this element have that class? Yes or no. Periodt.

html 💖

Gets or sets the innerHTML, bestie. It's giving 'let me peek inside' energy. 👀

my @html = $q->html();

my $html = $q->html(); # 1st matching element only

$q->html('<p>foo</p>');

insert_before 🦋

Puts every element in the set of matched elements before the target. Like, right before. No cap.

insert_after 💁‍♀️

Puts every element in the set of matched elements after the target. After, not before. We been knew.

prepend 🎀

Inserts content (the parameter you pass) to the beginning of each element in the set of matched elements. It's giving 'first in line' vibes.

remove 💀

Deletes the elements associated with the object from the DOM. Poof, gone.

# remove all <blink> tags from the document
$q->find('blink')->remove;

remove_class 🧚

Removes a single class, multiple classes, or literally all classes from each element in the set of matched elements. Class dismissed.

replace_with 👑

Replaces the elements of the object with the provided replacement. The replacement can be a string, a Web::Query object, or an anonymous function. The anonymous function gets passed the index of the current node and the node itself (which is also localized as $_). Iconic.

my $q = wq( '<p><b>Abra</b><i>cada</i><u>bra</u></p>' );

$q->find('b')->replace_with('<a>Ocus</a>);
    # <p><a>Ocus</a><i>cada</i><u>bra</u></p>

$q->find('u')->replace_with($q->find('b'));
    # <p><i>cada</i><b>Abra</b></p>

$q->find('i')->replace_with(sub{
    my $name = $_->text;
    return "<$name></$name>";
});
    # <p><b>Abra</b><cada></cada><u>bra</u></p>

size 🔢

Returns the number of elements in the Web::Query object. Like, how many are we talking?

wq('<div><p>foo</p><p>bar</p></div>')->find('p')->size; # 2

text 📝

Gets or sets the text. Pretty straightforward, bestie.

my @text = $q->text();

my $text = $q->text(); # 1st matching element only

$q->text('text');

If you call it in scalar context, it only returns the string representation of the first element. So don't be surprised. 🙈

OTHERS 🧾

Web::Query->last_response() — Returns the last HTTP response status that was generated by new_from_url(). It's giving receipts.

HOW DO I CUSTOMIZE USER AGENT? 💅

Bestie, you can specify your own instance of LWP::UserAgent. Slay.

$Web::Query::UserAgent = LWP::UserAgent->new( agent => 'Mozilla/5.0' );

FAQ AND TROUBLESHOOTING 🌈

How to find XML processing instructions in a document? ✨

Okay so it's possible with Web::Query::LibXML and by using an xpath expression with find():

# find <?xml-stylesheet ... ?>
$q->find(\"//processing-instruction('xml-stylesheet')");

But heads up: the support for processing instructions in HTML::TreeBuilder::LibXML::Node is sketchy, so methods like attr() won't work. It's giving buggy. 😭

Can't get the content of script elements 🐛

So the <script> tag is treated differently by HTML::TreeBuilder, the parser used by Web::Query. To get the content, you can use either the method html() (but caveat: the content will be escaped), or use Web::Query::LibXML, which parses the 'script' element differently. The more you know.

my $node = "<script>var x = '<p>foo</p>';</script>";

say Web::Query::wq( $node )->text;
    # nothing is printed!

say Web::Query::wq( $node )->html;
    # var x = &#39;&lt;p&gt;foo&lt;/p&gt;&#39;;

say Web::Query::LibXML::wq( $node )->text;
    # var x = '<p>foo</p>';

say Web::Query::LibXML::wq( $node )->html;
    # var x = '&lt;p&gt;foo&lt;/p&gt;';

INCOMPATIBLE CHANGES 😌

0.10

In 0.10, new_from_url() no longer throws an exception on a bad response from the HTTP server. So like, it's chill now.

AUTHOR 👑

Tokuhiro Matsuno <tokuhirom AAJKLFJEF@ GMAIL COM> — the icon, the legend.

LICENSE 📜

Copyright (C) Tokuhiro Matsuno

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. So like, do your thing, but keep it free. 💖

BUGS 🐛

Please report any bugs or feature requests on the bugtracker website <https://github.com/tokuhirom/Web-Query/issues>.

When you submit a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature. Help us help you. 🫶

This page is from Web::Query(3pm), perl v5.38.2, dated 2024-01-20. Stay slaying, bestie. 💅✨

📦 where this page comes from

debian trixie: libweb-query-perl 1.01-1 · Artistic-1.0-Perl OR GPL-1.0-or-later
ubuntu noble: libweb-query-perl 1.01-1 · Artistic-1.0-Perl OR GPL-1.0-or-later
ubuntu resolute: libweb-query-perl 1.01-1 · Artistic-1.0-Perl OR GPL-1.0-or-later
ubuntu stonking: libweb-query-perl 1.01-1 · Artistic-1.0-Perl OR GPL-1.0-or-later
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-10-14: a language model's rewrite of the original, under its terms

Web::Query(3pm)

💖 Web::Query 💖

✨ NAME ✨

✨ VERSION ✨

✨ SYNOPSIS ✨

✨ DESCRIPTION ✨

✨ FUNCTIONS ✨

✨ METHODS ✨

🌸 CONSTRUCTORS 🌸

🌈 TRAVERSING 🌈

add

contents

each

end

filter

find

first

last

match($selector)

not($selector)

and_back

map

parent

prev

next

next_until( $selector )