Fedora packaging tedium and bulk-rebuild
In a recent thread on Fedora’s development list, Miroslav Suchý asked about what Fedora packagers found most time-consuming, that could be simplified or better automated.
For me, personally, it’s rebuilding a stack of packages that Facebook develop - due to a lack of stable ABI (so it’s partly our own doing, yes), the entire stack has to be rebuilt each time.
There are some Fedora tooling that make this easier –
rpmdev-bumpspec can automate the process of updating the version and changelog on simple spec files, and
fedpkg request-side-tag lets any packager get a ‘side tag’ based off of an existing release, so packages can be built individually then merged back as a set, so there is no window in which the set of packages in, say, Rawhide, are inconsistent with each other, and there is no need to use buildroot overrides (which exposes the unreleased packages to other developers when they might still be buggy).
It’s still a tedious process though, and will get worse the more packages get added to this set. And I want the process to be reproducible so I can hand it off to someone else.
Enter bulk-rebuild. This is currently hard-coded to rebuild the set of Facebook packages I care about, but I hope to generalize it so it can be useful for others. Going to share this to the mailing list where it can hopefully drive some discussions.
This script is currently specific to bumping the FB packages that depend on Folly. It can be generalized but probably doesn’t make much sense unless the packages have matching version numbers.
src/fedora/fb-pkgs ❯ ./bulk-rebuild/bump-stack.sh 2020.12.28.00 /home/michel/src/fedora/fb-pkgs/folly ~/src/fedora/fb-pkgs/folly ~/src/fedora/fb-pkgs Switched to branch 'master' Current branch master is up to date. Downloading: https://github.com/facebook/folly/archive/v2020.12.28.00/folly-2020.12.28.00.tar.gz | 3.1 MiB Elapsed Time: -1 day, 16:00:01 Downloaded: folly-2020.12.28.00.tar.gz Could not execute new_sources: Request is unauthorized. Password for salimma@FEDORAPROJECT.ORG: Uploading: folly-2020.12.28.00.tar.gz ######################################################################## 100.0% Source upload succeeded. Don't forget to commit the sources file Done updating folly. Make any manual adjustments now then press ENTER [master 91ba7c3] Update to 2020.12.28.00 3 files changed, 6 insertions(+), 3 deletions(-) Enumerating objects: 9, done. Counting objects: 100% (9/9), done. Delta compression using up to 4 threads Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 614 bytes | 204.00 KiB/s, done. Total 5 (delta 2), reused 0 (delta 0), pack-reused 0 remote: Emitting a message to the fedora-messaging message bus. remote: * Publishing information for 1 commits remote: Sending to redis to log activity and send commit notification emails remote: * Publishing information for 1 commits remote: - to fedora-message To ssh://pkgs.fedoraproject.org/rpms/folly 1c2166c..91ba7c3 master -> master # other packages being updated ...
fedpkg request-side-tag and
fedpkg chain-build. If the release requested is not
f34 it would rebase from
If a side tag is passed as the second argument, it skips requesting and waiting for one, and assumes the side tag is ready to use.
wait-repo without specifying a build would always wait for the next repo build so it doesn’t make sense to wait.
The list of packages is currently hardcoded too, but it’s easy to fix.
src/fedora/fb-pkgs ❯ ./bulk-rebuild/build-stack.sh f32 Waiting for side tag: f32-build-side-35423 Successfully waited 3:18 for a new f32-build-side-35423 repo # repos being rebased to master Chain building watchman-2020.12.28.00-1.fc32 + [['git+https://src.fedoraproject.org/rpms/folly.git#d77413a204d813852e15a203e9298596db4e9cb9'], ['git+https://src.fedoraproject.org/rpms/fizz.git#489e3fad9c84b2ad9c4e0ac5a1999707fd615c7d'], ['git+https://src.fedoraproject.org/rpms/wangle.git#556764f3fffc18e862de5ca59fb6df7ddd325c4a'], ['git+https://src.fedoraproject.org/rpms/fbthrift.git#f526a69f4d0c24fa28097d32b45984e9c62d83da']] for f32-build-side-35423 Created task: 58576348 Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=58576348 Watching tasks (this may be safely interrupted)... 58576348 chainbuild (f32-build-side-35423): free
At the end of execution the user is prompted to create the Bodhi update. Once these made it to
updates-testing the side tag is automatically retired.
58576348 chainbuild (f32-build-side-35423) completed successfully Done! Run this to create your update: bodhi updates new --type enhancement --severity low --stable-karma 1 --from-tag f32-build-side-35423 --notes "Updating FB packages to ..."
Of note: the CLIs for
koji is currently meant for human, not machine, consumptions. Invoking them from Bash scripts and trying to use the output (e.g. getting the name of that side tag) involves some brittle parsing. I plan to rewrite this in Python anyway, but it might be useful to add an output formatter to some commands where it makes sense (e.g.
This post is day 9 of my #100DaysToOffload challenge. Visit https://100daystooffload.com to get more info, or to get involved.
Have a comment on one of my posts? Start a discussion in my public inbox by sending an email to ~email@example.com [mailing list etiquette]
Posts are also tooted to @firstname.lastname@example.org
You can use your Mastodon account to reply to this post.