GIRLREPO(1)

๐Ÿ“ฆ
this version ยท 7 versions
, in ubuntu questing

๐ŸŽ€ GIRLREPO(1) ๐ŸŽ€ โ€” from the girlpages Manual

okay so bestie, gather round, because girlrepo is THE tool and I need you to understand it ๐Ÿ’…โœจ


โœจ NAME โœจ

girlrepo โ€” build and publish girlpages repositories and their website, slay ๐Ÿซถ


๐ŸŒธ SYNOPSIS ๐ŸŒธ

girlrepo [-d dir] [-C file] ingest [options] path...
girlrepo [-d dir] ingest --system [options]
girlrepo [-d dir] crawl apt|rpm|pacman|apk|freebsd|nix|rfc url [suite]
[options]
girlrepo [-d dir] import [options] source
girlrepo [-d dir] unpack [options] [path]
girlrepo [-d dir] export [options] out
girlrepo [-d dir] girlify [options] [name...]
girlrepo [-d dir] publish [options]
girlrepo [-d dir] archive [options] archive
girlrepo [-d dir] gc | search | serve | status

like, yes, there's a lot of verbs, but honestly? iconic ๐Ÿ‘‘


๐Ÿ’– DESCRIPTION ๐Ÿ’–

So girlrepo does the whole pipeline, no cap: it collects man pages from packages and systems, girlifies them, and then publishes them as a repository โ€” which is literally just a tree of JSON files that any static web host can serve โ€” with a website sitting right on top ๐ŸŒธ. The way girl(1) can then fetch pages from a repository instead of asking a model? Obsessed. And the website lets anyone browse and search them, with a plain HTML page for every single man page so search engines can actually find them ๐Ÿ”ฅ.

Okay so, <hotpink>dir</hotpink> (which is just the current directory unless you pass -d) is the website. The repository itself lives in dir/repo. Upload the whole of dir and boom, you've published both ๐Ÿ’ซ. And listen: every command that changes the repository takes a lock on it, so they can totally be run from cron or CI without stepping on each other. main character energy, fr fr ๐Ÿฆ‹.

The usual cycle is like:

girlrepo ingest --distro debian --release trixie *.deb
girlrepo girlify
girlrepo publish --url https://example.org --gc

periodt. that's the vibe ๐Ÿ’….


๐ŸŒˆ COMMANDS ๐ŸŒˆ

๐Ÿ’œ ingest path...

Adds every English man page it finds in each path โ€” and bestie, path is flexible like that:

  • it can be a directory (a manual root such as /usr/share/man, or any tree containing one, like an unpacked package or a whole system image ๐Ÿ“),
  • or a package file: .deb, .pkg.tar.*, or .rpm (source packages are skipped, sorry ๐Ÿ™ˆ),
  • or the URL of one of those,
  • or a git repository: git+url, or any URL ending in .git, optionally followed by #ref for a branch, tag or commit.

Git repositories get cloned shallowly, and every roff page in them (name.section, perhaps compressed, starting with .TH or .Dt) is ingested wherever it is, apart from translations. (Pages generated at build time from other formats just aren't there to be found โ€” that's just how it is ๐Ÿ˜ญ.) Their package is the repository, at the tag or commit.

Packages are unpacked with tar(1) and cpio(1); neither dpkg nor rpm is needed โœจ. Translations (man/de/man1, and so on) are left out, we don't have time for them.

Each page is identified by the SHA-256 of its roff source, decompressed, with .so requests replaced by what they include, and symlinks followed (inside the unpacked package, for absolute links). The same page from different packages or distributions is stored once, noting everywhere it was found ๐ŸŽ€. A page whose source changed becomes a new version; old versions are kept (we don't throw away history, ever).

A page's current version โ€” the one the website shows โ€” is the latest of its main package, meaning the one the most families of systems have it in. FreeBSD and Nix only count if nowhere else has the page, since FreeBSD numbers its pages by its own release, and Nix's can be other programs of the same name ๐Ÿงš. The latest is by version number: the upstream version of the packages it was found in (1.35 from both 1.35+dfsg-3.1 and 1.35-5; epochs, Debian revisions, rpm and pacman releases and repacking suffixes don't count), then, for the same version, the one found in the most places, then the one added last ๐Ÿ’ซ. New versions get rendered with man(1) to make their man variant. And if girl(1)'s cache already has a conversion of the same text, that becomes the girl variant straight away โ€” instant slay โšก.

  • --system โ€” ingest this system's manpath instead of paths, noting which installed package (from the dpkg, pacman or rpm database) each page came from, and the distribution from os-release(5) ๐Ÿ’ป
  • --distro=id, --release=release โ€” record where the pages came from, such as debian and trixie
  • --package=name, --package-version=version, --arch=arch โ€” record the package, for directories, or for a single package file whose name doesn't follow its format's convention. (Package files are otherwise identified by their names, iykyk.)
  • -s, --sections=list โ€” only these sections, bestie
  • -j, --jobs=n โ€” render this many pages at once (default: one per CPU ๐Ÿ–ฅ๏ธ)
  • -n, --dry-run โ€” just say what would be added
  • --no-cache โ€” don't take conversions from girl's cache
  • -q, --quiet โ€” only report problems (we love a low-key run ๐Ÿ™ˆ)

๐Ÿฆ‹ crawl apt url suite | rpm url | pacman url | apk url | freebsd url | nix url | rfc [url]

Ingests every package in a repository that has man pages, or every RFC. Let's break it all the way down:

apt url suite

A Debian-style repository, such as https://deb.debian.org/debian trixie or http://archive.ubuntu.com/ubuntu noble ๐Ÿ“. Pages are recorded as from the suite's codename, without any -updates or -security (so bookworm-security is just bookworm). And lowkey a gotcha: Debian's security archive has empty Contents, so for a url ending -security the main archive's say which packages have man pages.

rpm url

An rpm-md (yum, dnf) repository: the directory holding repodata/.

pacman url

An Arch-style repository: the directory holding the database, such as https://geo.mirror.pkgbuild.com/core/os/x86_64.

apk url

An Alpine repository: the directory holding APKINDEX.tar.gz, such as https://dl-cdn.alpinelinux.org/alpine/v3.22/main/x86_64, with its release from the URL (v3.22 is 3.22). Its index lists no files, so the packages taken are those Alpine puts man pages in: -doc packages, and man-pages and man-pages-posix ๐Ÿ’….

freebsd url

A FreeBSD release's distribution sets: the directory holding their MANIFEST, such as https://download.freebsd.org/releases/amd64/15.1-RELEASE, with its release from the URL. The base system's pages (its commands, libraries, file formats and section 9's kernel interfaces) are taken from base.txz; third-party software's, in the pkg repositories, aren't, since their indexes list no files ๐Ÿ˜ญ.

nix url

A Nix channel, such as https://channels.nixos.org/nixos-25.05, with its store paths in the binary cache --nix-cache (https://cache.nixos.org by default), recorded as from nixos (or nixpkgs) and the channel's release (25.05, unstable), each package by the name and version in its store path ๐Ÿงš. The channel lists its store paths, and the cache, for each, its files, so every store path that could hold man pages (a man or bin output, or the main one) is looked at, at most --lookups at once (default 32), and those that do are fetched as NARs, of which only the man pages are unpacked. What each store path holds is remembered (they never change), in girlrepo's cache, so later crawls only look at new ones; the first looks at about 136,000, which takes about ten minutes โณ. Most packages split their man pages into their own output, but some keep them in their main one, which is fetched whole.

rfc [url]

RFCs, from the RFC Editor (or a mirror of https://www.rfc-editor.org/ at url), by its rfc-index.xml: each one with text, as page number in section rfc, its title its description, its text (cleaned of page breaks and running headers and footers) its original, and ietf its source ๐ŸŒธ. --rfcs numbers (comma separated) crawls just those. They're girlified with a prompt of their own, which keeps every BCP 14 key word (MUST, SHOULD, MAYโ€ฆ) and section number, and on the website live at rfc/number/.


okay so the big picture here, bestie, is that man pages only exist inside packages, compressed as a whole, so they can't be fetched on their own โ€” annoying, I know ๐Ÿ˜ค. Instead the repository's file lists (apt's Contents, rpm's filelists, pacman's .files; for Alpine, its naming) say which packages have English man pages, and only those are downloaded (checking their SHA-256 against the index, where it has them), ingested a few at a time with their package, version, architecture and license (from the index, or for Debian's, which don't say, its machine-readable copyright file), and then deleted ๐Ÿ—‘๏ธ. Each page's own license and copyright lines, from comments in its source, are kept too; crawling again with --again fills them in for pages ingested before they were. Only the newest version of each package is taken. What's been ingested is remembered (in ~/.cache/girlpages/girlrepo-crawl/), so crawling again only fetches packages that are new or changed โ€” efficiency, we love her ๐Ÿ’….

โš ๏ธ ngl real talk: repository signatures aren't checked; use https mirrors. no cap, be safe out there ๐Ÿ’–
  • --components=list, --arch=arch โ€” for apt: which components (default: all the suite has) and architecture (default amd64; architecture-independent packages come too)
  • --repo=name โ€” for pacman: the database name, if it isn't the directory two above the URL
  • --distro=id, --release=release โ€” what to record, if not what the repository says
  • -m, --match=regexp โ€” only packages whose names match
  • --limit=n โ€” stop after this many packages
  • -j, --jobs=n โ€” download this many packages at once (default 4)
  • --group=n โ€” ingest this many packages together (default 16); a page can only include pages from packages in its group
  • --again โ€” ingest packages even if an earlier crawl did
  • --all-versions โ€” take every version the repository has, not just the newest
  • --all โ€” download every package if the repository has no file lists
  • -n, --dry-run โ€” list the packages that would be fetched, and how much that is
  • -s, --no-cache, -q โ€” as for ingest ๐Ÿซถ

๐ŸŽ€ import source

Merges another published girlpages repository into this one, like a little repo playdate ๐ŸŒธ. source is a URL or a directory: the repository itself, or a site with it in repo/.

  • Pages this repository doesn't have are added whole ๐Ÿฆ‹
  • versions it doesn't have are added to its pages, keeping when and where they were found
  • versions both have gain each other's packages and any variants this one lacks โœจ

Pages whose listings are already identical are skipped, and every file fetched is checked against the hash its listing gives, so a damaged or tampered file is refused (and reported) rather than imported โ€” no sketchy files allowed, periodt ๐Ÿ’…. Run publish afterwards.

And if the other repository has packs (see publish), it's synced by them: the packs this repository hasn't had from it are fetched (a few big files, rather than a request for every one), checked, and unpacked into a copy of it kept in the cache, and the pages they changed merged from there. The next import fetches just the packs added since ๐Ÿ”ฅ.

  • --prefer-remote โ€” when both have a variant of the same version and they differ (two girl conversions, usually), take theirs; otherwise this repository's is kept ๐Ÿซถ
  • -s, --sections=list, -m, --match=regexp โ€” only these sections, or pages whose names match. ๐Ÿ’…
  • -j, --jobs=n โ€” fetch this many pages at once (default 8). โšก
  • -n, --dry-run โ€” say what would be imported. ๐Ÿซถ
  • --no-packs โ€” fetch its files one by one, even if it has packs. ๐Ÿ™ˆ
  • --remerge โ€” merge every page in its packs again, not just those that changed since the last merge. Pages whose listings are the same here are still checked: if any file a listing names is missing (a merge stopped part of the way, or a file system that lost what it was writing), it's fetched again. Any merge does that for the pages it merges; this one merges them all. publish leaves versions whose files are missing out of the search index, and says how many there are; this is what puts them back. ๐Ÿ”ฅ

unpack [path]

Merges packs that are already here into this repository, as import does with packs it fetches: those in path, a pack-only repository (as export writes) or a directory of pack files, or with no path, those dropped into dir/repo/packs/ (but not this repository's own). ๐Ÿ’–

Packs in dir/repo/packs/ are moved out of the way as they're unpacked; those in path are left as they are. It takes import's options, but for --no-packs. With --remerge, give the packs' path, as those moved out of dir/repo/packs/ aren't there any more. โœจ

export out

Writes the repository as packs into out, a pack-only repository: its packs, and a manifest.json listing them, for import or unpack elsewhere (or serving). If out has packs already, the changes since its last are added, as publish adds to the repository's own. ๐ŸŽ€

  • --pack-size=MiB โ€” how big full packs get (default 256). ๐ŸŒธ

girlify [name...]

Converts the current (latest) version of every page (or just the named ones: a name for any section, or name(section) for one) that has no girl variant yet, using the backend and model from girl's configuration (see girl(1)). Aliases sharing a source are converted once. Interrupting it is safe: each page is saved as it's done, and the next run carries on. ๐Ÿฆ‹

  • --variant=name โ€” make this variant instead of girl: the pages are rewritten in its voice, and each gets it alongside its others. --list-variants lists them: boy and femboy (who keep every detail, as girl does), lolcat and pirate (every detail, in a dialect), eli5 and woman (every detail, plus explanations or stories), and witch, bofh and bottom (every literal and warning, but perhaps only the gist of the rest). Only girl takes conversions from girl's cache, and each variant has its own prompt versions, for --reprompt. ๐Ÿ‘‘
  • --names-from=file โ€” only the pages listed in file, along with any named on the command line: one a line, as name or name(section), with # for comments, at the start of a line or after a space. It's how to give some pages more variants than the rest. A list with no names in it is an error, and a name with no page is reported. ๐Ÿ“
  • --provider=name โ€” use the provider configured as [providers.name] in girl's configuration (see girl(1)). ๐Ÿ”Œ
  • --backend=name, --model=model โ€” override the configured backend and model. ๐Ÿ› ๏ธ
  • -j, --jobs=n โ€” send this many requests at once (default from the configuration): to as many pages, or, when fewer pages are left, to the parts of long ones. Long pages report their progress every half minute, and each part is kept as it's converted, so a page that fails part way only needs its missing parts the next time. โฑ๏ธ
  • --limit=n โ€” stop after this many pages. ๐Ÿ›‘
  • -s, --sections=list โ€” only these sections. ๐Ÿ“š
  • --all-versions โ€” convert old versions too. ๐Ÿ•ฐ๏ธ
  • --batch=api, --no-batch โ€” send pages in batches with openrouter, anthropic or sail, or not at all, overriding the batch setting. --batch-requests n puts at most n requests in each batch (overriding batch_max_requests), so results come back in smaller lots. Batches work as for girl --bulk; see girl(1). What was sent is saved in ~/.cache/girlpages/girlrepo-batches/, and the next girlify collects whatever has finished. ๐Ÿ“ฆ
  • --no-wait โ€” send batches off and return without waiting. ๐Ÿƒโ€โ™€๏ธ
  • --repair=how โ€” what's done with a batch's parts that failed, or whose replies dropped options: batch (the default) asks them again in the next batch, at batch price, keeping a page whose reply dropped options as it is meanwhile (and replacing it if the next reply's better), and holding one with a part that failed until it has it (sent whole after three goes); live asks the live model there and then, at full price (which in a flex window can take a long while); none keeps replies that dropped options as they are, and sends a page with a part that failed whole in the next batch. ๐Ÿฉน
  • --reprompt โ€” convert again versions whose variant was made with an older prompt. Listings record each variant's prompt version, and status counts them. ๐Ÿ”
  • --force โ€” convert again, even pages that already have a girl variant. ๐Ÿ’ช
  • -n, --dry-run โ€” list what would be girlified, with a rough count of the requests and tokens it would take: about four bytes a token, and output --output-ratio (default 1.2, measured from real conversions) times as long as the original. Thinking tokens, for models that think, aren't counted. ๐Ÿงฎ
  • --price=in/out โ€” with --dry-run, estimate the cost at these US dollars per million input and output tokens, both live and in batches at half price. ๐Ÿ’ธ
  • --no-cache, -q, --quiet โ€” as for ingest. ๐Ÿคซ

publish

Writes the section indexes, the search index and manifest.json, then the website. Until it runs, nothing ingested or girlified since the last publish is visible. Website files are only rewritten when they change, so syncing the result somewhere is cheap. ๐ŸŒˆ

The website's static pages are what search engines should see: the home page, the section listings (in pages of 200, section/page/n/, which the browser scrolls through as one long list, keeping only what's near the screen), and each page's current version, girlified if it has been (otherwise the original), with canonical links, OpenGraph and Twitter card tags, JSON-LD (TechArticle and BreadcrumbList for pages, WebSite with a SearchAction for the home page, CollectionPage for listings), a sitemap, and llms.txt (what the site is and where to read it, for language models and their tools; see https://llmstxt.org). โญ

Each page's static page is at girl/section/name/ (girl/1/ls/, girl/rfc/9110/ for a series' document), girlified (or the original, until it is). In any other variant it's at variant/section/name/ (man/1/ls/ for the original, pirate/1/ls/): nothing's pre-rendered there, and servers send those (and listings, searches and the home page in a variant: pirate/section/1/, pirate/) the app's shell, static/shell.html, which the app draws the page in; listings are at section/key/ (section/1/, section/rfc/), and the packages pages come from, A to Z, at packages/ (from every source), packages/distro/ and packages/distro/release/, each package's pages at package/name/ (or, from one source, in the app: #/debian/trixie/package/coreutils), all of them scrolling through as one list, as section listings do. The header links to the sections (the home page) and the packages. serve and girlserve(1) know a site's variants from its manifest, and send a series' old addresses (rfc/9110/, rfc/) where they are now; dist/hosting/ has the same rules for nginx and static-web-server. A reader can make a variant their default: pages linked to (and girly pages opened) are in it from then on, where they have it. ๐Ÿ’–

Everything else happens in the browser, read straight from the repository by an app in the home page's hash: older versions (#/1/ls/9.4, by upstream version or the start of a source hash), and browsing by where pages come from, a distribution (#/debian), a release of one (#/debian/trixie/section/1, #/debian/trixie/1/ls) or upstream for git repositories, each showing the version of every page that source has. /search/ searches every page from every source, all of their versions' text, and can keep to one source; by words, or by parts of words, for phrases in double quotes, and within a package (pkg:coreutils or pkg:debian/coreutils, which on its own lists every page from it, as each page's "where this page comes from" links to). Without --url there are no absolute URLs, so no canonical links, OpenGraph URLs or sitemap. ๐Ÿ”Ž

  • --url=url โ€” the site's absolute URL, for canonical links and sitemap.xml. ๐ŸŒ
  • --base=path โ€” the URL path the site is served from, if not /, such as /girlpages/ for a GitHub Pages project site. ๐Ÿ“
  • --no-site โ€” only update the repository. ๐Ÿ—‚๏ธ
  • -j, --jobs=n โ€” write this many files at once (default one per CPU). On a network file system (CephFS, NFS), where each file is mostly a wait for the server, many more than that (32, say) can be much quicker. โšก
  • --no-packs โ€” don't add to the repository's packs. Otherwise, each publish adds the changes since the last to dir/repo/packs/: the repository's files (but not its indexes, which are worked out from them) as a log of packs, zstd-compressed tar files of about 256 MiB, for mirroring it without a request for every file. The changes since the last full pack are kept in small tail packs, which are replaced by full packs once they'd make one; full packs never change. The first publish packs everything, which takes a while. ๐Ÿ“ฆ
  • --packs=path โ€” list the packs in path (a directory of pack files, as unpack takes) as the repository's, in place of its own, adding nothing to them. For a repository made by unpacking just those packs, as a publishing job does from scratch each time: it's a copy of the one they came from, so its log is theirs, and publishing that keeps the site's packs the same from one publish to the next. ๐Ÿงš
  • --from-packs=path โ€” publish the repository that the packs in path make, plus its website, straight into --archive (which it literally needs), with no repository in dir, or anything else written anywhere: their files are replayed into the archive's new version as they're read, the indexes and website made from there, and the packs listed as the repository's (as with --packs). It's what unpack then publish --packs --archive make, without the millions of files in between, so a job publishing packs needs no scratch space. โœจ
  • --archive=dir โ€” add the website (and repository) to the archive in dir afterwards: see archive. The archive compresses what it adds, so --precompress defaults to none with it. It takes archive's options too.
  • --gc โ€” collect garbage afterwards. ๐Ÿงน
  • --precompress=list โ€” write compressed copies of the site's and repository's files, in these formats (default zstd,br,gzip; none for none): see HOSTING. Only new and changed files are compressed, and copies of removed files go.
  • --compress-jobs=n โ€” compress this many files at once (default: as many as --jobs).

The website is a progressive web app: it can be installed, and a service worker keeps the app itself (scripts, styles, and the browse, search and offline pages) for offline use. Repository files fetched by the browser are kept in its Cache API: content-addressed ones are read from there without asking the network, and the files that change are kept as a fallback for when there's no network. Offline, the service worker sends static pages to the browse app, which renders them from those files: every page read before, every page of a section saved with its listing's save for offline button, and any search, once the search page's search offline button has fetched the search index (and its typo matching offline button the index that catches typos), with a switch to show only the pages that can be read offline (on by default when offline). Every half hour while a page is open, the browser looks at manifest.json; when it has changed, the search index files and section indexes the new one doesn't use are deleted in idle time. ๐Ÿฆ‹

๐Ÿงน gc

Deletes files nothing refers to any more: superseded listings, section indexes and index shards, and the contents of versions no listing mentions. Content-addressed files may be cached by browsers and proxies forever, so it is kinder to collect garbage some time after publishing rather than straight away. -n says what would go. ๐Ÿ‘‘

๐Ÿ” search query...

Searches the published index the way the website does. --repo searches a repository elsewhere; -l sets how many results, and --ngrams finds the words query words are anywhere inside of (dir finds mkdir), as the website's "parts of words" does, rather than only the words they start. Words in double quotes are a phrase, which pages must have as it is (bar case and punctuation), and pkg:name keeps to pages from a package, as on the website.

๐Ÿ“ฆ archive archive

Adds the website in dir (and the repository in it) to the archive in archive (making it, if it's not there), as its new version: a few big files to serve the site from, made for a network file system, where a site's millions of small files are slow. It holds segments, files of up to a gigabyte of the site's files' bytes, each stored as it's sent (as it is, and compressed with zstd, brotli and gzip, where that's smaller); an index for each version, of where each file's bytes are; and current, naming the version being served, which serve --archive and girlserve(1) watch. Only files whose content isn't in the archive already are compressed and added: the rest of the new version points at where they are. Versions replaced are kept (at least --keep of them, and any newer than --keep-for), for servers still on one, and then they, and the segments only they used, go. The repository's contents and what's generated from them (its indexes and the website, which change each publish) are kept in segments of their own, and when less than half of either's bytes are in use, it's compacted: what is, is copied into new segments, and the other's left alone. One archive (or publish --archive) runs at a time. The format is in doc/archive-format.md in the source.

  • -j, --jobs=n โ€” compress this many files at once (default one per CPU).
  • --compact โ€” compact it now, whatever its state.
  • --keep=n, --keep-for=duration โ€” keep at least n versions (default 5), and any replaced less than duration ago (default 1h).

๐Ÿ–ฅ๏ธ serve

Serves dir (or, with --archive, an archive) at http://localhost:1990/ (or --listen). It serves the way a production server should (each file in the smallest form the client accepts, with its cache lifetime, ETags and the site's 404 page), and takes the options of girlserve(1), which is the same server on its own: --archive, --admin-listen, --reload-interval, --access-log=file (a file, or - for standard output), --drain and --drain-delay (default 0 here).

๐Ÿ“Š status

Counts pages, versions and conversions.


๐Ÿ’– REPOSITORY FORMAT ๐Ÿ’–

This is a summary: the full specification, for writing tools that read or write repositories, is doc/repo-format.md in the girlpages source.

Everything is JSON. Files named name-sha.json or sha.json are named after the SHA-256 of their own contents and never change, so they can be cached forever. manifest.json and the pointer files change in place and should be cached briefly. ๐ŸŽ€

  • manifest.json โ€” the root: sections, page counts and the index manifests.
  • contents/ab/source/variants.json โ€” every generation of every variant of one version of a page: each variant made again (with a new prompt, say) is a new generation, and the latest is the highest.
  • contents/ab/source/variant-sha.json โ€” one generation of a variant, named after its own hash.
  • page/name-section/name.json โ€” points at the page's listing.
  • page/page-ab/sha.json โ€” every version of a page, where each was found, and its variants' hashes.
  • section/section.json โ€” points at the section's index.
  • section/section-sha.json โ€” every page in a section, with its description and listing hash.
  • index/words/manifest-sha.json โ€” the range of words in each word shard.
  • index/words/shard-ab/sha.json โ€” words, each with the pages it appears in as [doc id, score].
  • index/grams/manifest-sha.json โ€” the range of trigrams in each gram shard.
  • index/grams/shard-ab/sha.json โ€” trigrams, each with the words containing it, for finding misspelt words.
  • index/docs/manifest-sha.json โ€” the shards of the page table that doc ids index.
  • index/docs/shard-ab/sha.json โ€” part of the page table: [name, section, description, sources].
  • index/variants/variant/manifest-sha.json โ€” the range of keys in each of a variant's index shards.
  • index/variants/variant/shard-ab/sha.json โ€” the first 16 hex digits of the source hash of every version with that variant, sorted, so a client like girl --bulk or girl --import can tell which of its pages a repository has without asking for each. manifest.json names each variant's index under index.variants, and lists the variants under variants.

source is the page's source hash (as above) and ab its first two characters. Variants are girl (girlpages markup, split into sections at top-level headings) and man (man's plain text, split at its headings, with the header and footer lines apart), and there may be others: any short lower-case name ([a-z][a-z0-9-]*), for pages rewritten with other prompts, each saying its syntax so clients know how to show it; joining a variant's sections gives back exactly the original. Names and sections are escaped in file names: anything but letters, digits and _+-.,:@ (or a leading dot) becomes %XX, which must be percent-encoded again in URLs. ๐ŸŒธ

A client with a page installed needs one request: repo/contents/ab/source/girl.json. The website's search reads the manifests, then only the shards a query needs.


๐ŸŒˆ HOSTING ๐ŸŒˆ

A site is plain files: any static web server will do. publish also writes compressed copies of every file worth compressing (.zst, .br and .gz; see --precompress), compressed as hard as each format goes, once, so the server only has to send the right one: static-web-server with --compression-static, or nginx with gzip_static, brotli_static and zstd_static. JSON shrinks to about a quarter. girl(1) asks for zstd, then brotli, then gzip, and browsers ask for what they can take. Ready-made configurations are in the source, dist/hosting/static-web-server.toml and dist/hosting/nginx.conf. ๐Ÿ’…

They send these Cache-Controls, as girlrepo serve does:

  • public, max-age=31536000, immutable โ€” files named after their own hash: listings (repo/page/page-*/), section indexes (repo/section/*-sha.json), variants (repo/contents/*/*/variant-sha.json), and everything under repo/index/.
  • public, max-age=300 โ€” repo/manifest.json, and the pointers into the rest (repo/page/name-*/, repo/section/section.json, repo/contents/*/*/variants.json), which change on every publish.
  • no-cache โ€” sw.js, so browsers pick up a new site.
  • public, max-age=3600 โ€” everything else.

A CDN such as CloudFront can go in front, keeping those lifetimes (set its cache policy to use the origin's headers) and passing Accept-Encoding through to the origin so its copies are used; CloudFront normalises that header to the encodings it knows, so check zstd is among them, or clients behind it will get brotli. Files a publish removes are best left until clients' caches have moved on: publish, sync, and collect garbage (gc) a few minutes later, rather than all at once. ๐Ÿซถ


๐ŸŽ€ FILES ๐ŸŽ€

  • dir/repo/ โ€” the repository.
  • dir/repo/.lock โ€” held while the repository is being changed.
  • dir/repo/packs/ โ€” the repository's packs, and index.json, which lists them.
  • ~/.cache/girlpages/girlrepo-sync/ โ€” copies of the repositories import and unpack have synced packs from: their packs, and what unpacking them gives.
  • ~/.cache/girlpages/girlrepo-parts/ โ€” parts of long pages girlify has converted, until their page is done.
  • dir/man/section/name/ โ€” a page's current version on the website.
  • dir/browse/ โ€” sends links from earlier versions of the browser app to where it is now.

๐Ÿ›‘ EXIT STATUS ๐Ÿ›‘

okay so the exit codes are giving very clear vibes, bestie:

  • 0 โ€” success, slay ๐Ÿ’…
  • 1 โ€” usage error, aka you typed something wrong
  • 2 โ€” something failed, no cap ๐Ÿ˜ญ
  • 130 โ€” interrupted

and Ctrl+C stops any command: the first press asks it to stop, which it does as soon as it's safe (a publish leaves the repository as it was published last), and a second press stops it right away. periodt.


๐Ÿ’– EXAMPLES ๐Ÿ’–

  • Start a repository from this machine, girlifying the first hundred pages.
girlrepo ingest --system && girlrepo girlify --limit 100 && girlrepo publish
  • Add every page from Arch's package cache.
girlrepo ingest --distro arch --release rolling /var/cache/pacman/pkg/*.pkg.tar.zst
  • Build for a GitHub Pages project site.
girlrepo publish --base /girlpages/ --url https://naomiamethyst.github.io/girlpages
  • See how much of Debian has man pages, and how big those packages are.
girlrepo crawl apt https://deb.debian.org/debian trixie --components main -n
  • Add tmux's page as of its 3.5a tag.
girlrepo ingest git+https://github.com/tmux/tmux#3.5a
  • Estimate what girlifying section 1 would cost.
girlrepo girlify -n -q -s 1 --price 0.10/0.50
  • Write the repository as packs, then on another machine, merge them into a repository of its own (and again later, fetching just what's new).
girlrepo export /srv/girlpages-packs
girlrepo -d mirror import https://example.org/girlpages-packs
  • Read ls(1) from a repository on disk.
girl --repo ./repo ls

๐Ÿ‘€ SEE ALSO ๐Ÿ‘€

go read these too, iykyk: girl(1), girlserve(1), man(1) ๐ŸŒธ


โœจ AUTHOR โœจ

Naomi Persephone Amethyst <naomi@amethyst.name> ๐Ÿ‘‘


that's the whole page, bestie โ€” source/version girlpages, date 2026-10-11, manual GIRLREPO(1). love u, byeee ๐Ÿ’–โœจ๐Ÿฆ‹

๐Ÿ“ฆ where this page comes from
  • ubuntu questing
  • ๐Ÿ“œ the page's license: GPL-3.0-or-later
  • ๐Ÿ’– girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms