EPEL Minor Version Mass Branching

Description

Originally, the EPEL repository had packages that built against the latest RHEL minor, but since EPEL10, the EPEL repository now builds against the latest Centos Stream release. To keep consistency for RHEL releases, EPEL repository goes through a mass branching process to generate a new minor release branch. This process let’s the developers continue to maintain their packages against Centos Stream without risking to break packages that already work on the current active RHEL minor target.

Send announcement

One day before the mass branching, we send out announcement because during mass branching, new koji builds for EPEL are disabled.

Remove Targets

In Fedora it is recommended to stop connections to koji through firewall. But it was decided not to require it for this process.

In the case of EPEL, just removing the targets is enough to disable it’s usage for the time being while the branching process it’s being done.

$ koji remove-target epel10-candidate
$ koji remove-target epel10

Cancel all running build for EPEL10

List all running tasks and select only those relevant for current branching.

$ koji list-builds --state=0 --type=rpm | grep el10_0 | awk '{print $1}'

Cancel each of those tasks

$ koji cancel <build>

dist-git

Now we need to update dist-git in two steps:

  • Create the new branch in git

For both of these actions you will need the list of active repos in EPEL10.

Get current EPEL active packages

To get the list of all the current active packages, you must run the scripts/branching-epe/get_all_active_packages_branching.sh script.

sh scripts/branching-epel/get_all_active_packages_branching.sh 10

The results will be saved to the components_epel10.1.txt

Create the git branches

On pkgs01.stg (for testing) or pkgs02 (for production), run:

$ sudo -u pagure python /usr/local/bin/mass-branching-git.py --branch-from  epel10 epel10.0 components_epel10.txt

The first argument is the new epel version and the second one is the path to the file generated in the previous step.

Ansible

Apps in ansible need to be updated to be aware of the new branch.

The following sections describe the things that need to be updated, but it’s recommended to use this PR for reference.

Bodhi

Bodhi needs to be updated to add new release. This needs to be done in bodhi2 role in infra ansible repo. This change includes, updating koji-sync-listener.py, new-updates-sync, pungi configs for rpm updates, bodhi templates.

  • roles/bodhi2/backend/files/new-updates-sync

  • roles/bodhi2/backend/tasks/main.yml

  • roles/bodhi2/backend/templates/koji_sync_listener.toml

In this step in Fedora, roles/bodhi2/backend/templates/pungi.rpm.conf.j2 would also need to be modified. But here it’s not required since the signing key doesn’t change between minor versions

Toddlers

Add new SLA to the toddlers App

Make sure that the SLAs are added. Check this PR for reference on what needs to be available.

Koji hub

Update the koji hub config to allow side tags for new koji epel10 minor (currently epel10.1) tag.

  • roles/koji_hub/templates/hub.conf.j2

Robosignatory

All minor branches use the same key, so it’s not required to generate a new one, but it’s important to update the configuration files so that the propper keys are used

  • roles/robosignatory/templates/robosignatory.toml.j2

Push the changes

When done editing the files, commit, push and apply them via the corresponding ansible playbooks:

$ sudo rbac-playbook groups/batcave.yml
$ sudo rbac-playbook groups/koji-hub.yml
$ sudo rbac-playbook groups/bodhi-backend.yml
$ sudo rbac-playbook manual/autosign.yml

Koji

The koji build system needs to have some tag/target work done to handle builds from the new branch and to update where builds from epel10 go.

Run make-koji-release-tags script in pagure releng repo

Make sure that things tagged on epel10.0-testing are tagged correctly. https://bodhi.fedoraproject.org/updates/?search=&status=pending&status=testing&releases=EPEL-10.0

Bodhi

Linking Empty Repos

We need to link empty repos so that new-updates-sync wont complain about missing repos. The following commands should be run on bodhi-backend01.phx2.fedoraproject.org

$ sudo ln -s /mnt/koji/compose/updates/empty-repo/ /mnt/koji/compose/updates/epel10.1
$ sudo ln -s /mnt/koji/compose/updates/empty-repo/ /mnt/koji/compose/updates/epel10.1-testing

Creating Empty Repos

To create empty repos on the master mirror, run create_empty_repos.sh from pagure releng repo. This should be run on bodhi-backend01.phx2.fedoraproject.org

$ sudo -u ftpsync sh scripts/branching-epel/create_empty_repos.sh 10.0

Creating EPEL10.1 release

To create a new minor release in bodhi, you need to run:

$ bodhi releases create \
    --name "EPEL-10.1" \
    --long-name "Fedora EPEL 10.1" \
    --version 10 \
    --branch epel10 \
    --id-prefix FEDORA-EPEL \
    --dist-tag epel10.1 \
    --stable-tag epel10.1 \
    --testing-tag epel10.1-testing \
    --candidate-tag epel10.1-testing-candidate \
    --pending-signing-tag epel10.1-signing-pending \
    --pending-testing-tag epel10.1-testing-pending \
    --pending-stable-tag epel10.1-pending \
    --override-tag epel10.1-override \
    --mail-template fedora_epel_legacy_errata_template \
    --state current \
    --composed-by-bodhi \
    --no-create-automatic-updates

Recreate targets

Once mass-branching is complete and the new release exists in bodhi, re-create these targets with the following commands:

$ koji add-target epel${release_major}           epel${release}-build epel${release}-testing-candidate
$ koji add-target epel${release_major}-candidate epel${release}-build epel${release}-testing-candidate

Updating MirrorManager

MirrorManager will load the changes automatically, you can check that everything is in order throught the following command:

$ curl 'https://mirrors.fedoraproject.org/metalink?arch=x86_64&repo=epel-10.0'

Branch new EPEL10 minor in Koschei

Branch new fedora EPEL10 minor in koschei.

Want to help? Learn how to contribute to Fedora Docs ›