rt-serializer(8)

💖 rt-serializer — the Request Tracker Reference's own little data-dumper 💖

okay so bestie, this is rt-serializer(8), and it's giving "pack up my whole entire database and mail it somewhere else" energy. Let's get into it. 🧚✨

---

🌸 NAME 🌸

rt-serializer — serialize an RT database to disk. literally writing your whole database out to files, no cap. 💾

---

🎀 SYNOPSIS 🎀

rt-validator --check && rt-serializer

---

✨ THE VIBE (what it actually does) ✨

So this script writes out the entire RT database to disk, so you can import it later into a different RT instance. cute, right? 🦋

But here's the thing: it requires that the data in the database be self-consistent in order to do that. So please, bestie, make sure the database you're exporting passes validation by rt-validator before you even think about running rt-serializer. Don't skip this. 🙈

While it's running, it'll try to estimate how many objects are left to serialize. Those estimates are pessimistic, and they will be wrong if --no-users, --no-groups, or --no-tickets are in play. so don't come for the progress bar, it's doing its best. 😭

Also! If your controlling terminal is big enough (more than 25 columns high) and the gnuplot program is installed, it'll show you a textual graph of the queue size over time. iconic behavior honestly. 📈✨

---

💅 OPTIONS 💅

• --directory name — the name of the output directory to write data files to. It should not exist yet, and it's a fatal error if it does. Defaults to ./$Organization:Date/, where $Organization is whatever is set in RT_SiteConfig.pm, and Date is today's date. 🗂️
• --force — removes the output directory before starting. yeet it and begin. 💥
• --size megabytes — by default rt-serializer chunks its output into data files around 32Mb in size; this option sets a different threshold size, in megabytes. Heads up: this is the threshold after which it rotates to writing a new file, so it's technically the lower bound on the size of each output file. math is math. 🍓
• --no-users — by default all privileged users get serialized; passing --no-users limits it to only the users referenced by serialized tickets and history, i.e. the ones that are necessary for internal consistency. 👑
• --no-groups — by default all groups get serialized; passing --no-groups limits it to only system-internal groups, the ones needed for internal consistency. 🫶
• --no-assets — by default all assets get serialized; passing --no-assets skips assets during serialization. 🌈
• --no-deleted — by default all tickets and assets, including deleted ones, get serialized; passing --no-deleted skips deleted tickets and assets during serialization. 💀
• --scrips — no scrips or templates are serialized by default; this option forces all scrips and templates to be serialized. 🔥
• --acls — no ACLs are serialized by default; this option forces all ACLs to be serialized. 🔐
• --no-tickets — skip serialization of all ticket data. 🎟️❌
• --limit-queues — takes a list of queue IDs or names separated by commas. When provided, only that set of queues (and the tickets in them) will be serialized. 🎯
• --limit-cfs — takes a list of custom field IDs or names separated by commas. When provided, only that set of custom fields will be serialized. 🧩
• --hyperlink-unmigrated — replaces links to local records which are not being migrated with hyperlinks. Those hyperlinks will use the serializing RT's configured URL. 🔗 Without this option, such links are instead dropped, and transactions which had updated such links get replaced with an explanatory message. lowkey sad but that's the deal.
• --clone — serializes your entire database, creating a clone. Use this if you want to migrate your RT database from one database type to another (e.g. MySQL to Postgres). It is an error to combine --clone with any option that limits object types serialized. No dependency walking is performed when cloning. rt-importer will detect that your serialized data set was generated by a clone. 🧬
• --incremental — will generate an incremenal serialized dataset using the data stored in your IncrementalRecords database table. (yes, the manpage says "incremenal", we're leaving it, it's part of the lore 🫠) This assumes you have created that table and run RT using the Record_Local.pm shim as documented in docs/incremental-export/. 📚
• --gc n — adjust how often the garbage collection sweep is done; lower numbers are more frequent. See GARBAGE COLLECTION below. 🧹
• --page n — adjust how many rows are pulled from the database in a single query. Disable paging by setting this to 0. Defaults to 100. 📄 Keep in mind that rows from RT's Attachments table are the limiting factor when determining page size. You should likely be aiming for 60-75% of your total memory on an otherwise unloaded box. 🧠
• --quiet — do not show graphical progress UI. 🤫
• --verbose — do not show graphical progress UI, but rather log was each row is written out. (again, the manpage's wording, we respect the source ✍️) 🗣️

---

🧹 GARBAGE COLLECTION 🧹

rt-serializer maintains a priority queue of objects to serialize, or searches which may result in objects to serialize. When inserting into this queue, it does no checking whether the object in question is already in the queue, or whether the search will contain any results. Those checks happen later — when the object reaches the front of the queue, or during periodic garbage collection. 🗑️✨

During periodic garbage collection, the entire queue is swept for:

• objects which have already been serialized,
• objects that occur more than once in the queue,
• and searches which contain no results in the database.

This is done to reduce the memory footprint of the serialization process, and it's triggered when enough new objects have been placed in the queue. This parameter is tunable via the --gc parameter, which defaults to running garbage collection every 5,000 objects inserted into the queue; smaller numbers mean more frequent garbage collection. 🔄

The default of 5,000 is roughly tuned based on a database with several thousand tickets, but optimal values will vary wildly depending on database configuration and size. Values as low as 25 have provided speedups with smaller databases; if speed is a factor, experimenting with different --gc values may be helpful. 🧪

Note that there are significant boundary condition changes in serialization rate, as the queue empties and fills, causing the time estimates to be rather imprecise near the start and end of the process. so like, don't panic. 😌

Setting --gc to 0 turns off all garbage collection. ⚠️ Be aware that this will bloat the memory usage of the serializer. Any negative value for --gc turns off periodic garbage collection and instead checks for objects already serialized or already in the queue at the time they would be inserted. 🚨

---

and that's the whole page, bestie!! this was rt-serializer(8) from the Request Tracker Reference, shipped with perl v5.38.2, dated 2023-10-29 — go serialize something and slay 💅✨🦋