MediaW...:Pages(3pm)

✨ MediaWiki::DumpFile::Pages(3pm) ✨

okay bestie, sit down, hydrate, because today we're reading MediaWiki::DumpFile::Pages — it's a Perl module from User Contributed Perl Documentation and it's literally your backstage pass to chewing through an XML dump file from a MediaWiki instance. no cap. 💖

---

🌸 NAME 🌸

MediaWiki::DumpFile::Pages — process an XML dump file of pages from a MediaWiki instance 🦋

---

💅 SYNOPSIS 💅

here's the vibe, in code, exactly as the ancestors wrote it:

use MediaWiki::DumpFile::Pages;

#dump files up to version 0.5 are tested
$input = 'file-name.xml';
#many supported compression formats
$input = 'file-name.xml.bz2';
$input = 'file-name.xml.gz';
$input = \*FH;

$pages = MediaWiki::DumpFile::Pages->new($input);

#default values
%opts = (
  input => $input,
  fast_mode => 0,
  version_ignore => 1
);

#override configuration options passed to constructor
$ENV{MEDIAWIKI_DUMPFILE_VERSION_IGNORE} = 0;
$ENV{MEDIAWIKI_DUMPFILE_FAST_MODE} = 1;

$pages = MediaWiki::DumpFile::Pages->new(%opts);
$version = $pages->version;

#version 0.3 and later dump files only
$sitename = $pages->sitename;
$base = $pages->base;
$generator = $pages->generator;
$case = $pages->case;
%namespaces = $pages->namespaces;

#all versions
while(defined($page = $pages->next) {
  print 'Title: ', $page->title, "\n";
}

$title = $page->title;
$id = $page->id;
$revision = $page->revision;
@revisions = $page->revision;

$text = $revision->text;
$id = $revision->id;
$timestamp = $revision->timestamp;
$comment = $revision->comment;
$contributor = $revision->contributor;
#version 0.4 and later dump files only
$bool = $revision->redirect;

$username = $contributor->username;
$id = $contributor->id;
$ip = $contributor->ip;
$username_or_ip = $contributor->astext;
$username_or_ip = "$contributor";

like… look at all those options. it's giving choose your own adventure but for dump files. 🌈

---

🎀 METHODS 🎀

💜 new

this is the constructor for this package, aka the main character of the whole thing. if you call it with a single parameter, that parameter has to be the input you want to parse — and that input is specified as either the location of a MediaWiki pages dump file, or a reference to an already-open file handle. either one works, we're flexible like that. 🫶

if you pass more than one argument to new, it has to be a hash of options. the keys are named:

🌟 input

this is the input to parse, as documented earlier. same energy, different seat.

🌟 fast_mode

have the iterator run in fast mode by default; defaults to false. scroll down to the section on fast mode below for the whole tea. 👑

🌟 version_ignore

do not enforce parsing of only tested schemas in the XML document; defaults to true.

---

💜 version

returns the version of the dump file. that's it, that's the whole method. simple. iconic.

---

💜 sitename

returns the sitename from the MediaWiki instance. ⚠️ requires a dump file of at least version 0.3.

---

💜 base

returns the URL used to access the MediaWiki instance. ⚠️ requires a dump file of at least version 0.3.

---

💜 generator

returns the version of MediaWiki that generated the dump file. ⚠️ requires a dump file of at least version 0.3.

---

💜 case

returns the case sensitivity configuration of the MediaWiki instance. ⚠️ requires a dump file of at least version 0.3.

---

💜 namespaces

returns a hash where the key is the numerical namespace id and the value is the plain text namespace name. the main namespace has an id of 0 and an empty string value. ⚠️ requires a dump file of at least version 0.3.

---

💜 next

accepts an optional boolean argument to control fast mode. if you pass that argument, it forces fast mode on or off. otherwise the mode is controlled by the fast_mode configuration option. (again: see the fast mode section below for the full lore.) 🦋

it is safe to intermix calls between fast and normal mode in one parsing session — so you can flip-flop like it's nothing.

in all modes, undef is returned if there is no more data to parse. that's your "we're done here" signal, bestie.

in normal mode, an instance of MediaWiki::DumpFile::Pages::Page is returned and the full API is available. 💅

in fast mode, an instance of MediaWiki::DumpFile::Pages::FastPage is returned; the only methods supported are title, text, and revision. this class can act as a stand-in for MediaWiki::DumpFile::Pages::Page — except it will throw an error if any attempt is made to access any other part of the API. so like, it's a stunt double, not a replacement. 💀

---

💜 size

returns the size of the input file in bytes. or, if the input you specified is a reference to a file handle, it returns undef.

---

💜 current_byte

returns the number of bytes of XML that have been successfully parsed.

---

🔥 FAST MODE 🔥

fast mode is a way to get increased parsing performance while dropping some of the features available in the parser. if you only require the titles and text from a page, then fast mode will decrease the amount of time required just to parse the XML file; some times drastically. ⚡

⚠️ heads up: when fast mode is used on a dump file that has more than one revision of a single article in it, only the text of the first article in the dump file will be returned; the other revisions of the article will be silently skipped over. silently!! no warning, no nothing. 😭

---

👑 MediaWiki::DumpFile::Pages::Page 👑

this object represents a distinct Mediawiki page and is used to access the page data and metadata. the following methods are available:

🌸 title

returns a string of the page title.

🌸 id

returns a numerical page identification.

🌸 revision

in scalar context returns the last revision in the dump for this page; in array context returns a list of all revisions made available for the page in the same order as the dump file. all returned data is an instance of MediaWiki::DumpFile::Pages::Revision.

---

🦋 MediaWiki::DumpFile::Pages::Page::Revision 🦋

this object represents a distinct revision of a page from the Mediawiki dump file. the standard dump files contain only the most specific revision of each page, and the comprehensive dump files contain all revisions for each page. the following methods are available:

✨ text

returns the page text for this specific revision of the page.

✨ id

returns the numerical revision id for this specific revision — and this is independent of the page id. don't get it twisted. 💅

✨ timestamp

returns a string value representing the time the revision was created. the string is in the format of "2008-07-09T18:41:10Z".

✨ comment

returns the comment made about the revision when it was created.

✨ contributor

returns an instance of MediaWiki::DumpFile::Pages::Page::Revision::Contributor.

✨ minor

returns true if the edit was marked as being minor, or false otherwise.

✨ redirect

returns true if the page is a redirect to another page, or false otherwise. ⚠️ requires a dump file of at least version 0.4.

---

💖 MediaWiki::DumpFile::Pages::Page::Revision::Contributor 💖

this object provides access to the contributor of a specific revision of a page. when used in a scalar context it will return the username of the editor if the editor was logged in, or the IP address of the editor if the edit was anonymous. 🫶

🌟 username

returns the username of the editor if the editor was logged in when the edit was made, or undef otherwise.

🌟 id

returns the numerical id of the editor if the editor was logged in, or undef otherwise.

🌟 ip

returns the IP address of the editor if the editor was anonymous, or undef otherwise.

🌟 astext

returns the username of the editor if they were logged in, or the IP address if the editor was anonymous.

---

💀 ERRORS 💀

🚨 E_XML_CREATE_FAILED — Error creating XML parser object

while trying to build the XML::TreePuller object a fatal error occurred; the error message from the parser was included in the generated error output you saw. at the time of writing this document the error messages are not very helpful, but for some reason the XML parser rejected the document. here's a list of things to check:

• make sure the file exists and is readable
• make sure the file is actually an XML file and is not compressed

🚨 E_XML_PARSE_FAILED — XML parser failed during parsing

something went wrong with the XML parser — the error from the parser was included in the generated error message. this happens when there is a severe error parsing the document, such as a syntax error. 😭

🚨 E_UNTESTED_DUMP_VERSION — Untested dump file versions

the dump files created by Mediawiki include a versioned XML schema. this software is tested with the most recent known schema versions and can be configured to enforce a specific tested schema. MediaWiki::DumpFile::Pages no longer enforces the versions by default, but the software author using this library has indicated that it should. when this happens it dies with an error like the following:

E_UNTESTED_DUMP_VERSION Version 0.4 dump file
"t/simpleenglish-wikipedia.xml" has not been tested with
MediaWiki::DumpFile::Pages version 0.1.9; see the ERRORS section of the
MediaWiki::DumpFile::Pages Perl module documentation for what to do at
lib/MediaWiki/DumpFile/Pages.pm line 148.

if you encounter this condition you can do the following:

1. Check your module version

the error message should have the version number of this module in it. check CPAN and see if there is a newer version with official support. the web page

http://search.cpan.org/dist/MediaWiki-DumpFile/lib/MediaWiki/DumpFile/Pages.pm

will show the highest supported version dump files near the top of the SYNOPSIS.

2. Check the bug database

it is possible the issue has been resolved already but the update has not made it onto CPAN yet. see this web page

http://rt.cpan.org/Public/Dist/Display.html?Name=mediawiki-dumpfile

and check for an open bug report relating to the version number changing.

3. Be adventurous

if you just want to have the software run anyway and see what happens, you can set the environment variable MEDIAWIKI_DUMPFILE_VERSION_IGNORE to a true value, which will cause the module to silently ignore the case and continue parsing the document. you can set the environment and run your program at the same time with a command like this:

MEDIAWIKI_DUMPFILE_VERSION_IGNORE=1 ./wikiscript.pl

⚠️ this may work fine, or it may fail in subtle ways silently — there is no way to know for sure without studying the schema to see if the changes are backwards compatible. you've been warned, bestie.

4. Open a bug report

you can use the same URL for rt.cpan.org above to create a new ticket in MediaWiki-DumpFile, or just send an email to "bug-mediawiki-dumpfile at rt.cpan.org". be sure to use a title for the bug that others will be able to use to find this case as well, and to include the full text from the error message. please also specify if you were adventurous or not, and if it was successful for you. 🫶

---

🎀 AUTHOR 🎀

Tyler Riddle, "<triddle at gmail.com>"

---

🐛 BUGS 🐛

please see MediaWiki::DumpFile for information on how to report bugs in this software.

💖 COPYRIGHT & LICENSE 💖

okay bestie, legal era incoming but it's actually cute, promise 🎀

Copyright 2009 "Tyler Riddle". yes the quotes are literally part of it, that's just how it was written, we don't question the vibe ✨

This program is free software — you can redistribute it and/or modify it under the terms of either:

1. the GNU General Public License as published by the Free Software Foundation; or
2. the Artistic License.

pick your fighter, both are valid, it's giving choose-your-own-license-adventure 💅

Want the full tea on these licenses? See http://dev.perl.org/licenses/ for more information. go read it, it's lowkey iconic 🦋

---

and that's a wrap on the whole page, bestie!! you made it to the end, main character behavior fr fr 👑✨

this lil masterpiece is perl v5.34.0, dated 2022-06-15 — shipped out under the MediaW...:Pages(3pm) title, courtesy of the User Contributed Perl Documentation crew 🫶

go forth and parse some pages, you absolute legend 💖🌸💅