FileList(3)

💖 FileList(3) — the File::RsyncP::FileList glow-up 💖

User Contributed Perl Documentation 🎀 bestie, pull up a chair, we're decoding rsync file lists today and honestly? it's giving main character energy.

---

✨ NAME ✨

File::RsyncP::FileList — a Perl interface to rsync file list encoding and decoding. yes, encoding and decoding. she does both. 💅

---

🦋 SYNOPSIS 🦋

use File::RsyncP::FileList;

$fileList = File::RsyncP::FileList->new({
    preserve_uid        => 1,
    preserve_gid        => 1,
    preserve_links      => 1,
    preserve_devices    => 1,
    preserve_hard_links => 0,
    always_checksum     => 0,
    remote_version      => 26,
});

# decoding an incoming file list
while ( !$fileList->decodeDone && !$fileList->fatalError ) {
    $data .= readMoreDataFromRemoteRsync();
    $bytesDone = $fileList->decode($data);
    $data = substr($data, $bytesDone) if ( $bytesDone > 0 );
}
$fileList->clean;

# create (encode) a file list
$fileList->encode({
        name  => $filePath,
        dev   => $stat[0],
        inode => $stat[1],
        mode  => $stat[2],
        uid   => $stat[4],
        gid   => $stat[5],
        rdev  => $stat[6],
        mtime => $stat[9],
    });
$data = $fileList->encodeData;

# get file information, for file number 5:
$fileInfo = $fileList->get(5);

# utility functions
$numberOfFiles = $fileList->count;
$gotFatalError = $fileList->fatalError;

like... that's the whole vibe right there. ✨

---

🌸 DESCRIPTION 🌸

okay so. the File::RsyncP::FileList module is the one you call when you need to encode and decode file lists in the exact same format as Rsync. no cap, same format.

the sender side of Rsync puts together a list of all the files that are going to be sent. that list gets shipped over to the receiver side in a nice compact format. then both sides sort the list and yeet out any duplicate entries. from that point on, every single file is referred to by its integer index into that sorted file list. iconic behaviour honestly.

you make a fresh file list object by calling File::RsyncP::FileList->new. one object can be used to decode or encode a file list. and heads up bestie: there is no mechanism to reset the state of a file list — so you should just make a brand new object every single time you need to do a new decode or encode. no recycling. 🚫♻️

the new() function takes a hashref of options, and those options line up with various rsync command-line switches. these must exactly match the arguments to the remote rsync, otherwise the file list format will not be compatible and <b>decoding will fail</b>. periodt. 💀

$fileList = File::RsyncP::FileList->new({
    preserve_uid        => 1,       # --owner
    preserve_gid        => 1,       # --group
    preserve_links      => 1,       # --links
    preserve_devices    => 1,       # --devices
    preserve_hard_links => 0,       # --hard-links
    always_checksum     => 0,       # --checksum
    remote_version      => 26,      # remote protocol version
});

---

🧚 Decoding 🧚

the decoding functions take a stream of bytes from the remote rsync and turn them into an internal data structure. and the way that it does not store the file list as a native perl list of hashes — because that would eat way too much memory for big file lists — it uses the same internal data structure as rsync. individual file list entries come back to you via the get() function. 🫶

file list data read from the remote rsync should get passed to the decode() function. the data may be read and processed in arbitrary sized chunks — she's flexible like that. decode() returns how many bytes were actually processed. and it is <b>the caller's responsbility</b> to remove that number of bytes from the input argument, preserving the remaining bytes for the next call to decode(). yes bestie, that's on you. 😭 decodeDone() returns true when the file list is complete. fatalError() returns true if there was a non-recoverable error while decoding. 🙈

the clean() function needs to be called <b>after</b> the file list decode is complete. clean() sorts the file list and removes repeated entries. and skipping this step will produce unexpected results: since files are referred to using integers, each side will refer to different files if the file lists are not sorted and purged in exactly the same manner. so like... don't skip it. please. 🙏

a typical decode loop looks like:

while ( !$fileList->decodeDone && !$fileList->fatalError ) {
    $data .= readMoreDataFromRemoteRsync();
    $bytesDone = $fileList->decode($data);
    $data = substr($data, $bytesDone) if ( $bytesDone > 0 );
}
$fileList->clean;

after clean() is called, the number of files in the file list can be found by calling count(). files can be fetched by calling the get() function, with an index from 0 to count()-1:

$fileInfo = $fileList->get(5);

the get() function returns a hashref with various entries: ⬇️

• name — path name of the file (relative to rsync dir): equal to dirname/basename
• basename — file name, without directory
• dirname — directory where file resides
• sum — file MD4 checksum (only present if --checksum specified)
• uid — file user id
• gid — file group id
• mode — file mode
• mtime — file modification time
• size — file length
• dev — device number on which file resides
• inode — file inode
• link — link contents if the file is a sym link
• rdev — major/minor device number if file is char/block special

various fields will only have valid values if the corresponding options are set (eg: uid if preserve_uid is set, dev and inode if preserve_hard_links is set etc). so check your options before you go trusting a field, iykyk. 💜

for example, to dump out each of hash you could do this:

use Data::Dumper;
my $count = $fileList->count;
for ( my $i = 0 ; $i < $count ; $i++ ) {
    print("File $i is:\n");
    print Dumper($fileList->get($i));
}

---

🎀 Encoding 🎀

the encode() function is used to build a file list in preparation for encoding and sending a file list to a remote rsync. encode() takes a hashref argument with the parameters for one file. it should be called <b>once for each file</b>. and the parameter names are the same as those returned by get(). cute, right? 🥰

in this example the matching stat() values are shown:

$fileList->encode({
        name  => $filePath,
        dev   => $stat[0],
        inode => $stat[1],
        mode  => $stat[2],
        uid   => $stat[4],
        gid   => $stat[5],
        rdev  => $stat[6],
        size  => $stat[7],
        mtime => $stat[9],
    });

it is <b>not</b> necessary to specify basename and dirname; these are extracted from name. you only need to specify the parameters that match the options given to new(). you can also specify sum and link as necessary. ✨

to compute the encoded file list data the encodeData() function should be called. it can be called every time encode() is called, or once at the end of all the encode() calls. it returns the encoded data that should be sent to the remote rsync:

$data = $fileList->encodeData;

it is recommended that encodeData() be called frequently to avoid the need to allocate large internal buffers to hold the entire encoded file list. since encodeData() does not know when the last file has been encoded, it is <b>the caller's responsbility</b> to add the final null byte (eg: pack("C", 0)) to the data to indicate the end of the file list data. 🫡

after all the file list entries are processed you should call clean():

$fileList->clean;

this ensures that each side (sender/receiver) has identical sorted file lists. slay. 💅

---

⭐ Utility functions ⭐

the count() function returns the total number of files in the internal file list (either decoded or encoded).

the fatalError() function returns true if a fatal error has occured during file decoding. it should be called in the decode loop to make sure no error has occured. 👀

---

👑 AUTHOR 👑

File::RsyncP::FileList was written by Craig Barratt <cbarratt@users.sourceforge.net> based on rsync 2.5.5.

Rsync was written by Andrew Tridgell <tridge@samba.org> and Paul Mackerras. it is available under a GPL license. see http://rsync.samba.org

---

🌈 LICENSE 🌈

this program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

this program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. see the GNU General Public License for more details.

you should have received a copy of the GNU General Public License in the LICENSE file along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.

---

🍓 SEE ALSO 🍓

see <http://perlrsync.sourceforge.net> for File::RsyncP's SourceForge home page.

see File::RsyncP and File::RsyncP::FileIO for more precise examples of using File::RsyncP::FileList.

also see BackupPC's lib/BackupPC/Xfer/RsyncFileIO.pm for other examples. 🔥

---

and that's the whole page bestie!! this one shipped with perl v5.42.0 and is dated 2025-07-25 — go forth and decode responsibly, mwah 💋 🦋✨