RDOPKG(1)

πŸ“¦
version 1.7.0 Β· 2 versions
, in epel 9

okay bestie, welcome to rdopkg(1) β€” yes that's the actual manual page name, and the whole thing is about rdopkg, the RDO packaging tool πŸ’– grab a snack, this one's a chonker.

✨ NAME ✨

rdopkg β€” RDO packaging tool 🌸

✨ SYNOPSIS ✨

rdopkg <action> <arg>...

rdopkg -c

rdopkg -h

✨ DESCRIPTION ✨

rdopkg is a tool for automating RDO/RHOSP packaging tasks, like introducing patches, updating packages to new versions and submitting packages to RDO. it literally does the tedious distgit homework so you don't have to πŸ˜­πŸ‘‘

Run rdopkg -h to get available actions. iykyk.


πŸŽ€ persistence πŸŽ€

rdopkg provides multi-step actions where each step is (should be) idempotent, so if something fails along the way or human interaction is required, rdopkg drops to shell, lets you fix the problem, and then continue by running rdopkg --continue (rdopkg -c). no starting over from scratch, we love a comeback arc πŸ’…

The state is stored in a file named .rdopkg.json in the current directory. The last stored action can be inspected (rdopkg status), continued (rdopkg --continue) or aborted (rdopkg --abort). And rdopkg will refuse to perform a new multi-step action if a state file is present β€” one thing at a time please, it's giving boundaries 🫢


πŸ”₯ ACTIONS πŸ”₯

🌈 Important actions diagram 🌈

okay this is the whole vibe of the tool in one flowchart, iconic honestly:

                    +---------------------+
                   /  WHAT DO YOU NEED?  /
                  +-----+----------+----+
                             |
                             |
                             |
                      build a new package
                             |
                             v
                    +---------------------+
                   /  what is changing?  /
                  +----+-----+-----+----+
                       |     |     |
                +------+     |     +--------+
                |            |              |
           just update   introduce    update to new
             .spec       patch(es)   upstream version
                |            |              |
                |            |              |
                v            v              v
             +-----+     +-------+    +-------------+
             | fix |     | patch |    | new-version |
             +-----+     +-------+    +-------------+

ACTION: fix

Apply changes to the .spec file. yes, THE file.

Flow:

  • Bump Release, prepare a new %changelog entry header.
  • Drop to shell, let user edit the .spec file.
  • After running rdopkg, ensure description was added to %changelog and commit changes in a new commit.

Example:

cd python-novaclient
rdopkg fix
vim python-novaclient.spec
rdopkg -c

More use cases can be found in rdopkg-feature-fix(7). πŸ¦‹


ACTION: patch

Introduce new patches to the package ✨

This action works with the patches branch (see AUTOMAGIC β†’ patches branch).

By default, rdopkg resets the local patches branch to the remote patches branch. You can skip this with -l/--local-patches to directly use the local patches branch as is.

Don't forget to git fetch before running the action, bestie, fr fr πŸ™ˆ

After running rdopkg patch, the new commit will contain the changes if there are any.

You can use -C/--changelog option to select how rdopkg detects new/old patches and changelog message:

  • detect: detect new/old patches using commit hash/subject (default)
  • count: count old and new patches (doesn't work for simultaneous removal & addition of patches and rewriting -patches history)
  • plain: just use generic - Update patches message

Or you can use -B/--no-bump to skip Release bump and changelog generation and only update patch files and their references in the .spec file. This is useful when you only need to align distgit with the patches branch. lowkey the chill mode πŸ’…

Use --amend to amend previous git commit with the changes and autogenerated commit message instead of creating a new one. This is very convenient when modifying distgit commits. slay.

Please see AUTOMAGIC for additional information about using magic patches_base and patches_ignore comments in your .spec file to influence patches generation.

Flow:

  • Unless -l/--local-patches was used, reset the local patches branch to the remote patches branch.
  • Update patch files from local patches branch using git format-patch.
  • Update .spec file with correct patch files references.
  • Unless -B/--no-bump was used, update .spec file: bump Release, create new %changelog entry with new patches' titles depending on -C/--changelog option.
  • If a %global commit asdf1234 macro declaration is present, rewrite it with the current sha1 of the patches branch. (This makes the sha1 value available during your package's build process. You can use this to build your program so that "mycoolprogram --version" could display the sha1 to users.) πŸ‘‘
  • Create new commit (or amend previous one with -a/--amend) with the changes using %changelog to generate commit message if available.
  • Display the diff.

Example:

rdopkg patch
rdopkg patch -lBa

More use cases can be found in rdopkg-feature-patch(7).


ACTION: update-patches

An alias for:

rdopkg patch --local-patches --no-bump

in the spirit of the ancient update-patches.sh script. vintage but make it cute πŸŽ€

See ACTION: PATCH above.


ACTION: new-version

Update package to new upstream version πŸš€

This action works with the patches branch (see AUTOMAGIC β†’ patches branch). After a successful rebase, rdopkg will offer to push the rebased patches branch.

Required new-version argument is a new version to rebase on, presumably a git version tag.

Don't forget to git fetch --all before running the action 😀

You can use the -N/--new-sources or -n/--no-new-sources options to control whether new-version will run fedpkg new-sources (rhpkg new-sources on Red Hat downstream products). By default, rdopkg will automatically enable this step in following scenarios:

  • Fedora distgit detected from origin git remote (pkgs.fedoraproject.org)
  • RH distgit detected from git branch (rhos-*, rh-*, ceph-*, rhscon-*)

Otherwise, fedpkg new-sources is disabled (-n).

After running rdopkg new-version, a new commit will contain the changes.

To only update .spec without touching patches branch, -b/--bump-only can be used. Along with -n/--no-new-sources this enables local-only operations, much like rpmdev-bumpspec:

rdopkg -bn

To note particular bugs in the changelog, use the -B/--bug option. rdopkg will append the supplied string to changelog in brackets. For example:

rdopkg new-version --bug rhbz#1234,rhbz#5678

will result in following %changelog line:

Update to 1.1.1 (rhbz#1234,rhbz#5678)

and corresponding Resolves: lines in commit message. receipts and everything 🧾✨

Flow:

  • Show changes between the previous version and the current one, especially modifications to requirements.txt.
  • Reset the local patches branch to the remote patches branch
  • Rebase the local patches branch on $NEW_VERSION tag.
  • Update .spec file: set Version, Release and patches_base to appropriate values and create a new %changelog entry.
  • Download source tarball.
  • Run fedpkg new-sources (rhpkg new-sources).
  • Update patches from the local patches branch.
  • Display the diff.

Example:

cd python-novaclient
git fetch --all
rdopkg new-version 2.15.0
# rebase failed, manually fix using git
rdopkg -c

More use cases can be found in rdopkg-feature-new-version(7).


ACTION: lint

Run checks for errors in current distgit. the vibe check for your package πŸ”

Available checks selectable with --lint-checks:

  • sanity: internal rdopkg sanity checks on the .spec
  • rpmlint: run rpmlint tool on the .spec
  • all: run all available checks (default)

Available error levels selectable with --error-level affect the exit code:

  • E: exit with code 23 when linting error is found (default)
  • W: exit with code 23 when linting error or warning is found
  • -: only print errors/warnings, always returns 0

Most of the time you probably want just:

rdopkg lint

Example of only running rpmlint with W error level:

rdopkg lint --lint-checks rpmlint --error-level W

ACTION: clone

Clone an RDO package distgit and setup remotes 🌈

clone uses rdoinfo metadata to clone the specified RDO package distgit and also setup relevant remotes to get you packaging quickly. literally the speedrun strat.

If your github username differs from your $USER, use -u/--review-user.

Example:

rdopkg clone -u github-user python-novaclient
cd python-novaclient
git remote -v

ACTION: query

Query RDO/distro repos for available package versions.

See rdopkg-adv-requirements(7) for complete example of query and other requirements management actions.

This action uses repoquery to discover the latest package versions available from RDO and other repos available on a supported distibution.

See output of rdopkg info for supported releases and distros.

Query specific RELEASE/DIST:

rdopkg query kilo/el7 openstack-nova

Query all dists of a release and show what's happening:

rdopkg query -v kilo openstack-nova

ACTION: reqquery

Query RDO/distro repos for versions defined in requirements.txt.

See rdopkg-adv-requirements(7) for a complete example of reqquery and other requirements management actions.

This action essentially runs rdopkg query on every module/package defined in requirements.txt and prints a colorful report to quickly find unmet dependencies. colorful report?? yes please 🎨 It accepts the same RELEASE/DIST filter as rdopkg query.

Python module names listed in requirements.txt are mapped to package names using the rdopkg.actionmods.pymod2pkg module.

Query requirements.txt from 2015.1 tag:

rdopkg reqquery -R 2015.1 kilo/el7

Query requirements.txt file:

rdopkg reqquery -r path/to/requirements.txt kilo/f21

Query .spec Requires (experimental):

rdopkg reqquery -s

Verbosely dump query results to a file and view them:

rdopkg reqquery -v -d
rdopkg reqquery -l

ACTION: reqcheck

Inspect requirements.txt vs .spec Requires. the showdown πŸ’€

See rdopkg-adv-requirements(7) for complete example of reqcheck and other requirements management actions.

This action parses the current requirements.txt from git and checks whether they're met in the .spec file. A simple report is produced.

Python module names listed in requirements.txt are mapped to package names using the rdopkg.actionmods.pymod2pkg module.

Use --spec/-s option to output Requires: suitable for pasting into .spec files. Version comparisons are hidden, whitespace is detected from .spec.

Use --strict/-S option to ask rdopkg to return an exit status. By default, 0 is returned.

Example:

rdopkg reqcheck
rdopkg reqcheck -s

Override file

okay so sometimes you literally need to keep reqcheck from looking at a Python module, or you need to change the module version sitting in requirements.txt so the whole reqcheck operation can pass. πŸ’… Some examples of this:

  • a Python module that got added to the requirements file but isn’t packaged in RDO yet β€” ignore it during that packaging window, bestie.
  • a Python module that got updated in the requirements file but isn’t packaged in RDO yet β€” replace that module version with the one specified in the .spec file.
  • an obsolete Python module that still lives in the requirements file but not in the .spec file gets tagged as MISSING in reqcheck output. It’s a false positive 😭 β€” the module should be removed upstream, but in the meantime, ignore it.
  • when a module RPM has several Provides for the same subpackage. Since pymod2pkg currently only supports one py3pkg name for the translation, this can give you a missing-module message even though the module is present under another name (e.g. PyYAML).

So there are only two operations, periodt:

  1. replacing a version of a Python module;
  2. ignoring (i.e. deleting) a Python module.

The adding operation is not handled. Packages that are in the .spec file and not in the requirements file are tagged as EXCESS during reqcheck, which does not make the reqcheck operation fail. That’s why adding a Python module to the requirements file before reqcheck isn’t relevant. 🫢

To ignore a Python module or overwrite a version during reqcheck, you need the --override/-O option and you provide a YAML file. The format looks like this:

---
packages:
  all:
    - name: "python-yaml"
    - name: "python-pbr"
      version: ""
  openstack-murano:
    - name: "python-alembic"
      version: ">= 0.9.6"
    - name: "python-pbr"
      version: ">= 2.0.0"

The first level describes the package name we want to override (e.g. openstack-murano). The second level includes the configuration of Python modules we want to replace or ignore. ✨

The keyword all is reserved in the first level. The configuration (list of Python modules) associated with all is applied to all packages. This can be interesting when a rule is spotted several times (e.g. PyYAML) β€” there’s a high chance that rule might apply every time. You declare it globally in all and then there’s no need to explicitly specify it in a package configuration. However, if a rule with the same Python module name is found in a specific package (e.g. openstack-murano) and in the all configuration, then the rule in the specific package configuration is picked up. Note: all configuration should be used carefully because those associated rules are applied globally. πŸ‘‘

In this example, for the package named openstack-murano:

  • rdopkg will replace alembic with alembic>=0.9.6 in the requirements file before comparing it to the alembic Requires version in the .spec file.
  • there is no version attribute for python-yaml, which is in the all configuration, so it will be ignored during reqcheck. The behavior is the same when the version attribute has an empty value.
  • python-pbr with an empty version in the all configuration means it is ignored during reqcheck. But since a rule with the same Python module name exists in openstack-murano, python-pbr>=2.0.0 is picked up during reqcheck.
Note: This option does not actually write to disk when replacing or ignoring Python modules in the requirements file. reqcheck loads this file as a memory stream, then the --override option replaces or ignores the modules provided in the YAML file in the stream. Then that stream is compared against the .spec file. πŸ’€

Example:

rdopkg reqcheck --override override-file.yml

ACTION: reqdiff πŸ¦‹

Show a pretty diff of requirements.txt.

See rdopkg-adv-requirements(7) for a complete example of reqdiff and other requirements management actions. πŸ“š

Use this to see how requirements changed between versions.

See the diff between the current and latest upstream version (automagic):

rdopkg reqdiff

See the diff between the current and a specified version:

rdopkg reqdiff 2015.1

See the diff between two supplied versions:

rdopkg reqdiff 2015.1 2015.2

ACTION: kojibuild πŸ”₯

Build the package in koji.

Flow:

  • Run the equivalent of fedpkg build using the, uh, disgusting fedpkg Python module. 😭
  • Watch the build.

Example:

rdopkg kojibuild

ACTION: amend 🌸

Amend the last git commit with current dist-git changes and (re)generate the commit message from %changelog.

This simple action is equivalent to running:

git commit -a --amend -m "$AUTOMAGIC_COMMIT_MESSAGE"

See AUTOMAGIC β†’ commit message for more information about the generated commit message. ✨

ACTION: squash πŸŽ€

Squash the last git commit into the previous one. The commit message of the previous commit is used.

This simple action is a shortcut for:

git reset --soft HEAD~
git commit --amend --no-edit

This is useful for squashing commits created by lower-level actions such as update-patches. 🫢

ACTION: get-sources πŸ’–

Download the package source archive.

Currently, Source0 from the .spec file is downloaded.

ACTION: info 🌈

Show information about RDO packaging.

Use this command to find out about:

  • currently supported RDO OpenStack releases
  • which distros are supported for each release
  • what branch to build from
  • what build system to build in
  • supported packages
  • various repositories tied to a package
  • package maintainers

This command is a human interface to rdoinfo. πŸ‘‘

Releases/dists/branches overview:

rdopkg info

Detailed information about a package:

rdopkg info novaclient

Filter packages by maintainers:

rdopkg info maintainers:jruzicka

Reverse filtering packages by not containing a tag:

rdopkg info tags:~newton

ACTION: release πŸ’…

Show information about RDO releases.

This command can be used to find out about:

  • display a list of all currently developed releases
  • see their status, branch, repos, source branch
  • filter information about a single release
  • print releases in a specified phase

To display info about all releases:

rdopkg release

Show only a specific release:

rdopkg release -r zed

Print releases in development:

rdopkg release -s development

Currently used statuses are: development, maintained, extended_maintenance.

Show releases that have repos for CentOS Stream 9:

rdopkg release -R el9s

ACTION: info-tags-diff ⭐

Show rdoinfo tag changes.

rdopkg info-tags-diff RDOINFODIR

will show per-package new/changed tags in rdoinfo between HEAD~..HEAD.

For an existing or new package, a list of changed tags is returned.

Example:

$ rdopkg info-tags-diff ~/.rdopkg/rdoinfo
openstack-changed ['newton-uc', 'newton']
openstack-new-pkg ['under-review']

This is an interface to rdopkg.actionmods.rdoinfo:tags_diff(). πŸ¦‹

ACTION: attr-tags-diff 🧚

Show rdoinfo attribute changes.

rdopkg info-attr-diff RDOINFODIR ATTRNAME [INFO_FILE]

will show per-package new/changed attributes in rdoinfo between HEAD~..HEAD.

For an existing or new package, a list of changes in the attribute is returned.

Example:

$ rdopkg info-attr-diff ~/.rdopkg/rdoinfo upstream
openstack-changed 'https://github.com/openstack-changed-repo'
openstack-new-pkg 'https://github.com/openstack-new-pkg'

If ATTRNAME is not used anywhere in the distroinfo repository, or there are no changes in the attributes, the command will return "No attribute changes detected." and exit with RC=0.

You can optionally pass INFO_FILE as a parameter to define the top-level YAML file to use.

This is an interface to rdopkg.actionmods.rdoinfo:attr_diff(). πŸ’œ

ACTION: findpkg πŸ”Ž

Find and show the single best matching package in rdoinfo.

This command produces the same output as rdopkg info but:

  • smart search is performed on package name, project name and upstream URL
  • only a single matching package is shown

Use -s/--strict to disable magic substring search and only match whole fields. ✨

This command is a human interface to distroinfo.query.find_package() function.

Examples of usage:

rdopkg findpkg nova
rdopkg findpkg -s openstack-nova
rdopkg findpkg git://git.openstack.org/openstack/nova
rdopkg findpkg openstack/nova
rdopkg findpkg novacli

ACTION: conf πŸ“

Display rdopkg’s local configuration.

This command prints the default configuration that ships with rdopkg out of the box. You can override the individual settings here by using .py files in the configuration directories.

Store your per-user configuration in ~/.rdopkg/conf.d/*.py, or store system-wide configuration in /etc/rdopkg.d/*.py. πŸŽ€

ACTION: tag-patches 🌟

Tag the local -patches branch with the package’s Name-Version-Release.

Since the -patches branch can change over time β€” including rebases, rewrites, etc. β€” we need a mechanism to keep historical records of what the -patches branch looked like over time. Tagging the -patches branch for each new NVR will maintain Git references to each snapshot of the particular patches that went into each build. 🫢

To look at the -patches branch for an old build, you can simply git checkout name-version-release for that build and get an exact representation of the Git tree for that build.

If a previous tag exists with this name, rdopkg will exit with an error unless you use the --force option to overwrite the existing tag with this name.

You can automatically push the new tag with the --push option. It’s a good idea to create and push the tag after every successful build. πŸ’–

✨ AUTOMAGIC ✨

Okay so here's the iconic part: instead of making you write project config files or type out an endless list of command line arguments, rdopkg literally just guesses all the necessary variables. She's intuitive like that. πŸ’…

🌸 patches branch

update-patches is the core lower level action for updating the dist-git .spec file with patches from the associated patches branch. rdopkg tries hard to detect the patches branch automagically β€” it's usually $BRANCH-patches for $BRANCH dist-git, but one patches branch per multiple dist-gits is also supported. Queen of flexibility fr fr. πŸ‘‘

Best illustrated by example (love that for us): the following are all valid patches branches for rhos-5.0-rhel-7 dist-git, and they're searched in that order:

  • rhos-5.0-rhel-7-patches
  • rhos-5.0-rhel-patches
  • rhos-5.0-patches ←-- preferred for RHOSP
  • rhos-patches

Use rdopkg pkgenv to check the detected patches branch. πŸ•΅οΈβ€β™€οΈ

You can specify the remote patches branch with the -p/--patches-branch action parameter for actions that use it, such as patch and new-version.

You may explicitly set the name of your patches remote and patches branch in your git configuration using the rdopkg.<branch>.patches-remote and rdopkg.<branch>.patches-branch options. For example, if you're working on a dist-git branch named rhel-7.4 and you want to use rhel-7-patches for your patches branch, you would run:

git config rdopkg.rhel-7.4.patches-branch rhel-7-patches

πŸ’– patches base

rdopkg calculates the git tag on which you are applying patches from the Version tag in your .spec file. If your .spec file contains a macro named milestone, the value of this macro will be appended to the version. So, if your spec file has:

Version: 2014.2.3

then rdopkg will use 2014.2.3 as the base. But if instead your .spec file has:

%global milestone rc2

Version: 2014.2.3

then rdopkg will use 2014.2.3rc2 as the base. Cute, right? πŸŽ€

In older versions of rdopkg, it was necessary to explicitly set the patch base using a special patches_base comment in your spec file. This is now optional behavior (the patches base is calculated automatically), but you can use this if you need to override the automatic behavior.

The most common use of patches_base is to specify the number of patches on top of the patches base (which defaults to spec Version) to skip:

# patches_base=+2

You can set an arbitrary git revision as a patches base:

# patches_base=1.2.3+2

You shouldn't need to modify this by hand (other than perhaps the number of skipped patches) because rdopkg manages patches_base as needed. Let her cook. 🍳

πŸ¦‹ patches_ignore

update-patches also supports filtering out patches based on matching a regex provided by a magic #patches_ignore comment in the spec file. This is useful, for example, in case the patches branch contains changes that are related to the CI/code review infra β€” useful to keep around, but they don't need to end up in the RPM. iykyk ✨

For example, if you add the following comments in your package's .spec file:

# patches_base=10.2.5
# patches_ignore=DROP-IN-RPM

then rdopkg will not create .patch files for any commits that have "DROP-IN-RPM" in the Git commit log's subject line.

Note: these lines should be directly above any Patch000X lines in your .spec file. Don't be chaotic about it bestie. πŸ™ˆ

πŸ”₯ Release bumping

rdopkg fix and rdopkg patch bump the Release tag in the .spec file.

By default, the last numeric-only part of Release is bumped:

1.1.1.a.b.c -> 1.1.2.a.b.c

You can override this using the -R/--release-bump-index argument, which expects MAJOR/MINOR/PATCH or an integer index of the release part to bump, starting at 1 from the left:

-R 1 / -R MAJOR: 1.1.1 -> 2.1.1
-R 2 / -R MINOR: 1.1.1 -> 1.2.1
-R 3 / -R PATCH: 1.1.1 -> 1.1.2
-R 4: 1.1.1.1 -> 1.1.1.2
-R 5: 1.1.1.1.1 -> 1.1.1.1.2
...

DLRN 0.date.hash and 0.1.date.hash formats are detected and default to bumping the 2nd Release part (-R 2). slay 🫢

πŸ‘‘ commit message

Commit messages created by rdopkg are generated from the .spec file Name, Version and Release (NVR) as well as the last %changelog entry. It's giving ✨ attention to detail ✨

All rdopkg actions that modify distgit use the following format:

package-name-1.2.3-4

Changelog:
- Doom the World (rhbz#111111)
- Fix Impending Doom support
- Save the World (rhbz#222222)

Resolves: rhbz#111111
Resolves: rhbz#222222
Change-Id: deadbeedeadbeedeadbeedeadbeedeadbeedeadbee

For each (rhbz#XYZ) mentioned in the latest %changelog entry, a Resolves: rhbz#XYZ line is appended to the commit message as required by the RHOSP workflow. πŸ’―

protip: to (re)generate a nice commit message after modifying the .spec file, use rdopkg amend (see ACTION: amend above). You're welcome. 😌

For example, the following %changelog entry:

%changelog
* Tue Feb 11 2014 Jakub Ruzicka <jruzicka@redhat.com> 0.5.0-1
- Update to upstream 0.5.0
- Fix evil Bug of Doom (rhbz#123456)

will generate the following commit message:

package-name-0.5.0-1

Changelog:
- Update to upstream 0.5.0
- Fix evil Bug of Doom (rhbz#123456)

Resolves: rhbz#123456

🌈 rdoinfo

rdoinfo is a special utility repository with RDO metadata:

https://github.com/redhat-openstack/rdoinfo

rdopkg uses rdoinfo to

  • detect release/dist from branch name
  • check valid RDO updates
  • query packages from RDO/distribution repos

and more. She does it all, no cap. πŸ’œ

You can view the rdoinfo metadata using rdopkg info. Go look, it's lowkey mesmerizing.


🌟 SEE ALSO 🌟

rdopkg-adv-new-version(7), rdopkg-adv-requirements(7), rdopkg-feature-pkgenv(7), rdopkg-feature-fix(7), rdopkg-feature-patch(7), rdopkg-feature-new-version(7), rdopkg-feature-actions(7)

πŸ’Œ CONTACT πŸ’Œ

rdopkg is maintained by Jakub Ruzicka <jruzicka@redhat.com>.

Bugs are tracked as github Issues:

https://github.com/softwarefactory-project/rdopkg/issues

To report a new bug:

https://github.com/softwarefactory-project/rdopkg/issues/new


That's the whole page bestie, you made it to the end!! πŸŽ‰πŸŒΈ This was RDOPKG(1) straight from the 10/04/2023 edition β€” go forth and slay your dist-git, and remember: when in doubt, rdopkg pkgenv and vibe check. πŸ’…βœ¨πŸ¦‹

πŸ“¦ where this page comes from