GFPREP(1)

✨ gfprep(1) ✨ — the Gfarm girlie's guide to parallel replication 💅

okay bestie, welcome to gfprep, part of the Gfarm manual set. this is the page that tells you how to make copies of your files everywhere at once and honestly? it's giving main character energy. let's get into it 🦋

---

🌸 NAME 🌸

gfprep — replicate Gfarm files in parallel ✨

---

🎀 SYNOPSIS 🎀

gfprep [-nqvdpPBU] [-mx] [-X regexp] [-S source-domainname]
       [-h source-hostfile] [-D destination-domainname]
       [-H destination-hostfile] [-j num-of-processes]
       [-J num-of-processes] [-M total-byte] [-z minimum-byte]
       [-Z maximum-byte] [-w way-of-scheduling]
       [-W kilobytes-for-threshold] [-s KB/s-to-simulate]
       [-F num-for-readahead] [-I sec-to-update] [-N num-of-replicas]
       path

like yes, that's a lot of flags. but every single one of them is there for a reason and we're gonna go through them all, no skipping 💖

---

💫 DESCRIPTION 💫

So gfprep creates replicas of Gfarm files in parallel. periodt.

When the path is actually a directory? bestie, it goes recursive — everything under that directory gets replicated too. it's giving thorough. 🌈

And path can be either a Gfarm URL or a path on a mount point of gfarm2fs. whichever vibe you're on, we support it.

Now, the host situation: you can specify a set of source/destination hosts using a domain name and/or a hostlist file. And if you give it both a domain name and a hostlist file? the set of hosts gets determined by both conditions together — intersection behavior, iconic. And when a set of source hosts is specified, only files stored on those source hosts are the ones getting replicated. no stray files, no chaos. 💅

Less busy file system nodes get picked for source or destination — the algorithm really said "I'm not doing the most, I'm doing the smartest." The maximum number of parallel replications/copies per host is the number of CPUs, so yes, your cores are the limit. if you wanna know about the number of CPUs, check the manual page of gfhost (iykyk).

Oh and also — gfprep retrieves the directory entries in parallel too. because why would anything be sequential, honestly. 🔥

---

👑 GFPREP OPTIONS 👑

these are the options that are only for gfprep — the exclusive club, the VIP section:

• -N num-of-replicas — tells it how many file replicas you need. and here's the slay part: if there are already a sufficient number of replicas in the destination hosts, neither any copying nor any removal will be done. it just... chills. if you don't specify this option, -N 1 is assumed.
• -x — removes extra file replicas when a set of destination nodes has more file replicas than specified by the -N option. it's the cleanup crew. 🧹
• -m — migrates file replicas from a set of source nodes to destination nodes. and if you specify it twice like -mm, it leaves the source file replica behind (so it's more of a copy-copy situation at that point).

⚠️ This option requires at least one of -S, -h, -D, and -H options.

⚠️ This option and the -N option are mutually exclusive. you literally cannot have both, bestie, pick a lane 💀

---

🌟 COMMON OPTIONS 🌟

There are several common options shared with gfprep — go peek at man gfpcopy for the whole set. (see man gfpcopy) 🫶

---

🍓 EXAMPLES 🍓

To replicate files under the directory recursively.

$ gfprep -N 3 gfarm:///dir

$ cd /mnt/gfarm2fs
$ gfprep -N 3 dir

To replicate a file or remove surplus replicas of the file.

$ gfprep -N 3 -x gfarm:///dir/file

To migrate replicas from gfsd1.example.com to other nodes.

$ gfprep -m -S gfsd1.example.com gfarm:///

---

🙈 NOTES 🙈

When the -L or -m option is not specified, a source node is not always selected within a set of specified source file system nodes. so like, don't assume, okay? 😭

It is better to execute the gfprep command near the metadata server. If you need to run it far from the metadata server, increase the parallelism with the -j and -J options. distance = latency = sadness, so crank those up. 💜

---

💀 SEE ALSO 💀

gfrep(1), gfpcopy(1), gfhost(1), gfarm2.conf(5)

---

and that's the whole page, bestie!! this was Gfarm, version dated 14 Jan 2017, GFPREP(1) — go forth and replicate responsibly ✨💖🦋