XML::F...r::Sort(3pm)

💖 XML::Filter::Sort 💖

okay bestie, welcome to XML::Filter::Sort(3pm) — a User Contributed Perl Documentation page about a SAX filter that sorts elements in XML. yes, sorting. in XML. while streaming. it's giving main character energy and honestly? we're obsessed. 🦋

---

🌸 NAME 🌸

XML::Filter::Sort — a SAX filter for sorting elements in XML 💅

---

✨ SYNOPSIS ✨

okay so here's the whole vibe in code form:

use XML::Filter::Sort;
use XML::SAX::Machines qw( :all );

my $sorter = XML::Filter::Sort->new(
  Record  => 'person',
  Keys    => [
               [ 'lastname',  'alpha', 'asc' ],
               [ 'firstname', 'alpha', 'asc' ],
               [ '@age',      'num',   'desc']
             ],
);

my $filter = Pipeline( $sorter => \*STDOUT );

$filter->parse_file(\*STDIN);

Or, if you're feeling feral and just want the command line:

xmlsort

periodt. 💅

---

💜 DESCRIPTION 💜

This module is a SAX filter for sorting 'records' in XML documents — and yes, that includes documents bigger than your available memory, no cap. 🧚

There's also a xmlsort utility that ships with this distribution so you can sort an XML file straight from the command line without writing a single line of Perl. If you want the deets on that, go read perldoc xmlsort. 🎀

---

🎀 EXAMPLES 🎀

okay so these examples all assume you're gonna make an XML::Filter::Sort object and drop it into a SAX::Machines pipeline, exactly like the synopsis up there. Could you use the object directly by hooking it up to a SAX generator and a SAX handler? Absolutely you could — but those details are omitted from the sample code, so don't come for me. 🙈

When you create an XML::Filter::Sort object (via the new() method), you must use the 'Record' option to say which elements you want sorted. The simplest way to do that is just... use the element name. like so:

my $sorter = XML::Filter::Sort->new( Record  => 'colour' );

Which could be used to transform this XML:

<options>
  <colour>red</colour>
  <colour>green</colour>
  <colour>blue</colour>
<options>

into this:

<options>
  <colour>blue</colour>
  <colour>green</colour>
  <colour>red</colour>
</options>

Want a more specific path to the record? Just add a prefix of element names separated by forward slashes, like:

my $sorter = XML::Filter::Sort->new( Record  => 'hair/colour' );

That one would only sort <colour> elements sitting directly inside a <hair> element — which means our sample document above would be left completely unchanged, bestie. and a path starting with a slash is an 'absolute' path, so it has to spell out every single intervening element from the root element all the way down to the record elements.

A record element may contain other elements. The sorting process can change the order of the record elements, but the order of any child elements inside them? nope, untouched. 🤚

The default sort uses the full text of each 'record' element and does an alphabetic comparison. Want more control? Use the 'Keys' option to give a list of elements inside each record whose text content should be used as sort keys. You can also use it to say whether keys get compared alphabetically or numerically, and whether the order comes out ascending or descending. like:

my $sorter = XML::Filter::Sort->new(
  Record  => 'person',
  Keys    => [
               [ 'lastname',  'alpha', 'asc'  ],
               [ 'firstname', 'alpha', 'asc'  ],
               [ '@age',      'alpha', 'desc' ],
             ]
);

Given this record ...

<person age='35'>
  <firstname>Aardvark</firstname>
  <lastname>Zebedee</lastname>
</person>

... the code above would use 'Zebedee' as the first (primary) sort key, 'Aardvark' as the second sort key, and the number 35 as the third sort key. So in this case, records with the same first and last name get sorted from oldest to youngest. slay. 👑

And just like with the 'record' path, you can specify a path to the sort key elements (or attributes). To make a path relative to the record element itself, start it with './'. 💫

---

💅 OPTIONS 💅

• Record => 'path string' — A simple path string defining which elements get treated as 'records' to be sorted (peep "PATH SYNTAX" below 👀). Elements that don't match this path will not be altered by the filter, at all, not even a little. Elements that do match will be re-ordered depending on their contents and on the value of the Keys option.

When a record element gets re-ordered, it takes its leading whitespace along with it. it's bringing its whole entourage. ✨

Also: only lists of contiguous record elements get sorted. If a list of records has a 'foreign body' in the middle — that's a non-record element, non-whitespace text, a comment, or a processing instruction, sitting between two elements — then it gets treated as two separate lists, and each one gets sorted in isolation from the other. no crossover, no drama. 🚧

• Keys => [ [ 'path string', comparator, order ], ... ]
• Keys => 'delimited string' — This option says which parts of the records should be used as sort keys. The first form uses a list-of-lists syntax, where each key is defined with a list of three things:
  1. The 'path string' — defines the path to an element or an attribute whose text contents get used as the value of the sort key (again, see "PATH SYNTAX" 💫).
  2. The 'comparator' — defines how those values get compared. It can be the string 'alpha' for alphabetic, the string 'num' for numeric, or a reference to a subroutine that takes two parameters and returns -1, 0 or 1 (similar to the standard Perl sort function, but without the $a, $b magic).

This item is optional and defaults to 'alpha'. 💖

3. The 'order' — should be 'asc' for ascending or 'desc' for descending, and if you omit it, it defaults to 'asc'.

Maybe you'd rather define the Keys with a delimited string than a list of lists? iconic, do you. Keys in the string get separated by either newlines or semicolons, and the components of a key get separated by whitespace or commas. One thing though: it is not possible to define a subroutine reference comparator using the string syntax. 💀

• IgnoreCase => 1 — Turn this on and sort comparisons become case-insensitive (instead of the default, which is case-sensitive). 🎀
• NormaliseKeySpace => 1 — The sort key values for each record are the text content of the child elements you specified with the Keys option (above). Enable this option and leading and trailing whitespace gets stripped from the keys, plus every internal run of spaces gets collapsed down to a single space. The default value for this option is off, for efficiency reasons. (we love a lazy default, ngl.)

Note: the contents of the record are not affected by this setting — only the copy of the data that's used in the sort comparisons. your actual XML stays pristine, bestie. 🫶

• KeyFilterSub => coderef — You can also supply your own custom 'fix-ups' by passing this option a reference to a subroutine. That subroutine gets called once for each record and gets handed a list of the key values for that record. The routine must return the same number of elements every time it's called, but that number may be less than the number of values passed in. You might use this option to combine multiple key values into one (eg: using sprintf).

Note: You can enable both the NormaliseKeySpace and the KeyFilterSub options together — space normalisation happens first. ✨

• TempDir => 'directory path' — This option does two jobs: it enables disk buffering instead of the default memory buffering, and it lets you say where on disk the data gets buffered. Heads up though — disk buffering is slower than memory buffering, so don't ask for it if you don't actually need it. More details live in "IMPLEMENTATION".

Note: It is safe to specify the same temporary directory path for multiple instances, since each one creates a uniquely named subdirectory (and cleans it up afterwards). 🌸

• MaxMem => bytes — Disk buffering mode actually sorts chunks of records in memory before saving them to disk. The default chunk size is 10 megabytes. Use this option to set a different chunk size (in bytes) that's better attuned to your available resources (more is better, obviously 💅). A suffix of 'K' or 'M' is recognised as kilobytes or megabytes respectively.

If you haven't enabled disk buffering (via 'TempDir'), the MaxMem option does absolutely nothing. And attempting to sort a large document using only memory buffering may result in Perl dying with an 'out of memory' error. that's not the vibe. 😭

• SkipIgnorableWS — If your SAX parser can do validation and generates ignorable_whitespace() events, you can enable this option to discard those events. If you leave this option at its default value (which implies you do want the whitespace), the events get translated into characters() events. 🦋

---

🌈 PATH SYNTAX 🌈

A simple element path syntax gets used in two places:

1. with the 'Record' option, to define which elements should be sorted
2. with the 'Keys' option, to define which parts of each record should be used as sort keys

In both cases you can use just an element name, or a list of element names separated by forward slashes. eg:

Record => 'ul/li',
Keys   => 'name'

If a 'Record' path begins with a '/' then it's anchored at the document root. If a 'Keys' path begins with './' then it's anchored at the current record element. Unanchored paths can match at any level. ⭐

A 'Keys' path can include an attribute name prefixed with an '@' symbol, eg:

Keys   => './@href'

Every element or attribute name can include a namespace URI prefix in curly braces, eg:

Record => '{http://www.w3.org/1999/xhtml}li'

If you don't include a namespace prefix, then all elements with the specified name get matched, regardless of any namespace URI association they might have. iykyk. 🧚

And if you include an empty namespace prefix (eg: '{}li') then only records that do not have a namespace association get matched. 💜

---

(part 1 done — more incoming, don't go anywhere bestie 🫶)

🌸 IMPLEMENTATION 🌸

okay bestie, so how does this thing actually sort stuff? 👀 It uses buffering. The tea:

• It does not need to buffer the entire document, thank u very much.
• BUT — for any sequence of records inside a document, all of those records have to be buffered. no skipping, no shortcuts, that's just the vibe.
• Unless you say otherwise, those records get buffered in memory.

And the memory it eats? lowkey a lot, ngl. The requirements are similar to DOM implementations — we're talking 10 to 50 times the character count of the source XML. 😭

So if your documents are big enough that you'd never even think about running them through a DOM parser, bestie… you need to enable disk buffering. Like, please.

---

💾 Disk buffering mode 💾

When you turn disk buffering on:

1. Sequences of records get assembled into 'chunks' of approximately 10 megabytes (and yes, that value is configurable, slay).
2. Each chunk gets sorted and saved to disk.
3. At the end of the record sequence, all the sorted chunks get merged together and written out as SAX events.

it's giving merge-sort realness, honestly 💅

🧠 Memory buffering mode 🧠

In memory mode, each record is represented as an XML::Filter::Sort::Buffer object, and it uses XML::Filter::Sort::BufferMgr objects to manage the buffers. For the full internals tea, go read XML::Filter::Sort::BufferMgr. 🔍

💿 Disk buffering mode (the internals) 💿

In disk mode, each record is represented as an XML::Filter::Sort::DiskBuffer object, managed by XML::Filter::Sort::DiskBufferMgr objects. Want the deep lore? That's XML::Filter::Sort::DiskBufferMgr. ✨

---

💀 BUGS 💀

okay so this is a real one, not me being dramatic:

ignorable_whitespace() events shouldn't be translated to normal characters() events — and maybe in a later release they won't be. 🙈

so like, heads up, that's the current behaviour, not the dream behaviour. fr fr.

---

👑 SEE ALSO 👑

XML::Filter::Sort requires XML::SAX::Base, and it plays so nicely with XML::SAX::Machines. 🫶

---

🎀 COPYRIGHT 🎀

Copyright 2002-2005 Grant McLean <grantm@cpan.org>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

---

and that's the whole page bestie!! 💖✨ You just read the XML::Filter::Sort(3pm) man page from the User Contributed Perl Documentation, served up as perl v5.26.1 and dated 2018-03-30 — go forth and sort ur records, u absolute icon 👑🌸💅