Scrapp...Parser(3pm)

✨ Scrappy::Scraper::Parser ✨

okay so bestie, welcome to the Scrappy::Scraper::Parser page 💖 this is the 3pm section of the User Contributed Perl Documentation, and it's literally the data-extracting little icon of the Scrappy framework. The official NAME blurb calls it a Scrappy Scraper Data Extrator — yes, spelled exactly like that, we don't make the rules, we just respect the source 💅

---

💜 VERSION 💜

version 0.94112090 — she's a classic, she's vintage, she's still serving 🎀

---

🌸 SYNOPSIS 🌸

here's the vibe, copy-paste ready:

#!/usr/bin/perl
use Scrappy::Scraper::Parser;

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->html($html);

    # get all links in all table rows with CSS selector
    my  $links = $parser->scrape('table tr a');

    # select all links in the 2nd table row of all tables with XPATH selector
    my  $links = $parser->scrape('//table/tr[2]/a');

    # percision scraping !
    # select all links in the 2nd table row ONLY with CSS selectors and focus()
    my  $links =
        $parser->select('table tr')
           ->focus(2)
           ->scrape('a');

(and yes, "percision scraping" is a direct quote from the original and honestly? iconic. no cap 🫶)

---

🦋 DESCRIPTION 🦋

Scrappy::Scraper::Parser hands you a whole lil toolkit for scraping and extracting info out of web pages, all inside the Scrappy framework. That's it, that's the whole job, and she does it well ✨

🎀 ATTRIBUTES 🎀

these are the object attributes that every single Scrappy::Scraper::Parser instance shows up with. bestie squad of four 🌈

data

data gets/sets the extracted data that came back out of the scrape method.

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select('table tr');
    $parser->data;

html

html gets/sets the HTML content that's gonna get parsed and extracted from. it's the raw material, the clay, the canvas 🎨

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->html($HTML);

html_tags

html_tags gets you a hashref of every known HTML tag and attribute, the ones meant to be used with Web::Scraper. it's the lore library 📚

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->html_tags;

worker

worker holds the Web::Scraper object — the actual engine doing the parsing and the extracting. she's the one clocking in 💼

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->worker;

---

🌟 METHODS 🌟

🍓 filter

the filter method lets you narrow down which tags come back in your results. you hand it a list of tag attributes you specifically want, and it forsakes everything else — including the special text and html tags/keys. it's giving selective, it's giving boundaries 💅

# filter results and only return meta tags with a content attribute
my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select('meta');
    print $parser->data;

    ...

    {
        name => '...',
        text => '...',
        html => '...',
        content => '....',
        http => '...',
        ....
    }

    print $parser->filter('name', 'content')->data;

    ...

    {
        name => '...',
        content => '....',
    }

look at that glow-up: from a whole pile of keys down to just name and content. obsessed 😭

🦋 focus

focus is how you zero-in on a specific block of HTML so your selectors only extract data from inside that highlighted block. use it after select has pulled rows of data — you pass it an array index and it locks tf in on that row. main character energy, but for one specific row 👑

my  $parser = Scrappy::Scraper::Parser->new;

# percision scraping !
# select all links in the 2nd table row ONLY
my  $links =
    $parser->select('table tr')
       ->focus(2)
       ->scrape('a');

🌈 scrape

scrape extracts data from the HTML you gave it and returns that extracted data. it's identical to the select method in every way except for what gets returned.

my  $parser = Scrappy::Scraper::Parser->new;
my  $links = $parser->scrape('a', $from_html); #get all links

🧚 select

select extracts data from the specified HTML and returns the parser object itself. then you hit up the data method to actually access the extracted info. chainable queen 🔗

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select('a', $from_html); #get all links

my  $links = $parser->data;

⭐ first

first returns the first element from the extracted dataset. the eldest, the head of the table 🥇

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select('a', $from_html); #get all links

my  $first_link = $parser->first;

# equivalent to ...
my  $first_link = $parser->data->[0];

💜 last

last returns the last element from the extracted dataset. the baby of the family, still valid 🥰

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select('a', $from_html); #get all links

my  $last_link = $parser->last;

# equivalent to ...
my  $last_link = $parser->data->[(@{$parser->data}-1)];

🎀 select_first

select_first is a convenience feature that smashes select() and first() together to return the first element from the extracted data. two icons, one method 🫶

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select_first('a'); #get link text
    $parser->select_first('a', 'href'); #get link URL

🌸 select_last

select_last is the same deal but it's select() + last() — returning the last element from the extracted data. the finale, the last bite 🍰

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select_last('a'); #get link text
    $parser->select_last('a', 'href'); #get link URL

🔥 each

each loops through the extracted dataset. it takes exactly one argument — a code reference — and each extracted item gets passed to it. it's giving for-loop but cuter 💫

my  $parser = Scrappy::Scraper::Parser->new;
    $parser->select('a', $from_html); #get all links

    $parser->each(sub{
        print shift->{href} . "\n"
    });

💖 has_html

has_html returns a boolean telling you whether HTML content has actually been set. vibe check for your parser 🌈

my  $parser = Scrappy::Scraper::Parser->new;
    print 'oh no' unless $parser->has_html;

lowkey important: if you never set the HTML, has_html is gonna be false and your scraping is not gonna scrape, bestie 😭

---

👑 AUTHOR 👑

Al Newkirk <awncorp@cpan.org> — thank you for your service, king 🫡

---

💌 COPYRIGHT AND LICENSE 💌

This software is copyright (c) 2010 by awncorp.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

---

and that's the whole page bestie, no notes 💅 this was perl v5.34.0, dated 2022-06-17, straight out of the Scrapp...Parser(3pm) corner of the docs. go forth and extract, you absolute legend 🌟🦋✨