This is some evidence these vhosts are impacted. Mitigate that with our
rules.
While we are at it we modify the ruleset to add a newly noticed item.
Change-Id: I8c20193e4e474898a0bdc395b25fd9de94469dd6
These docs had lived on the server in question in a text file as they
were sort of cobbled together from emails. Since then multiple renewals
have been successfully performed so we may as well add the process to
our actual documentation.
Change-Id: I13267ad08c1e4ef6007e5cbea040c274ea2f27d5
Almost immediately after we upgraded to 1.21.8 a new 1.21.9 release
became available. Again this appears to largely be a bugfix release with
no super important changes for us. However, there are performance
improvements which are always nice to see. The template files that we
override have not changed between 1.21.8 and 1.21.9.
Full change log can be found here:
https://github.com/go-gitea/gitea/blob/v1.21.9/CHANGELOG.md
Change-Id: Ica763081203d9be44c9de0923a261afa820c891b
This is a bugfix release with no template updates and no other impactful
deployment changes that I can see. Full changelog notes can be found
here:
https://github.com/go-gitea/gitea/blob/v1.21.8/CHANGELOG.md
Change-Id: I6009bbebc261e87702b7f603bf179be89d31edb9
This should cleanup our mirror update server so that we no longer have
configes (cron, scripts, logrotate rules, etc) for mirroring opensuse.
It won't clean up the afs volume, but we can get to that later (and it
will probably require manual intervention). This cleanup is done in a
way that it should be able to be applied to future cleanups too (like
when centos 8 stream goes away and everything is centos stream
specific).
Change-Id: Ib5d15ce800ff0620187345e1cfec0b7b5d65bee5
There are a number of issues with opensuse mirroring content cleanup
that this change aims to address. First up we fix the prefix for the
CentOS 7 networking content; it needed a repositories/ prefix. At the
same time we don't bother deleting the leaf data and instead delete the
more top level directory since we're cleaning this all up.
We then apply this top level cleanup to all of the repositories,
distributions, and updates. This is largely a noop (just some directory
removals) except in the case of update/ which still contains leap 15.2
update packages. These were apparently missed in the initial opensuse
cleaup.
After this lands we should end up with a largely empty volume.
Change-Id: Ic854fcecd1a0fabc388640a33da7e4e1f9ec07c0
We have removed CentOS 7 from nodepool now we can stop mirroring
pacakges for it. This deletes official CentOS 7 package mirror content
and OBS packages mirrored by the OpenSUSE mirror script for CentOS 7.
A followup change will remove the OpenSUSE mirroring entirely as this
was the last thing it was used for.
Change-Id: I484651b0845eaab933e98106684e0a2a6215b3d7
The clouds.yaml and rackdns config files do not need to use two
different Ansible vars to refer to the same credentials. Note that
the forward DNS account is separate, and so we still keep those
intact.
Change-Id: I9dd657f357d32083f2cfd7f074ba0d122ca803c3
After this merges, the temporary credential set opendevci_rax_*
and opendevzuul_rax_* can be removed from hostvars.
Depends-On: https://review.opendev.org/911163
Change-Id: I2e9067aa2f11100d311c86beb4df5bf15c72db69
Rackspace is requiring multi-factor authentication for all users
beginning 2024-03-26. Enabling MFA on our accounts will immediately
render password-based authentication inoperable for the API. In
preparation for this switch, add new cloud entries for the provider
which authenticate by API key so that we can test and move more
smoothly between the two while we work out any unanticipated kinks.
Change-Id: I787df458aa048ad80e246128085b252bb5888285
During the debian buster mirror cleanup we lost a volume backing afs on
afs01.dfw.openstack.org. Our existing docs gave us a good starting point
for recovery, but they could use more specifics. Add that info.
Change-Id: Ib334759314f0fd493e9b1bc8c06a8060ba8917ee
We are currently running MariaDB 10.4 for refstack. We use the
MARIADB_AUTO_UPGRADE flag to automatically upgrade the mariadb install
to 10.11 when switching the image version over to 10.11. This was
successfully performed against the lodgeit paste service.
Change-Id: I75262bc8eba3dd59d5869be9bf568fd66dc7f608
This includes a few extra steps that are needed to more fully cleanup
reprepro mirrors when we drop distro releases from reprepro. Without
this we leave some vestiges of old releases behind which can be
confusing in the future when we think we have already cleaned this stuff
up.
Change-Id: I15032314c39279999fbd6be74e9d73b76843399c
This upgrades our gitea container image and, thus deployment, to version
1.21.7 from 1.21.5. There are no updates to the three template files we
override upstream according to git diff in the gitea repo.
A full changelog can be found here:
https://github.com/go-gitea/gitea/blob/v1.21.7/CHANGELOG.md
Change-Id: I95d92f47085532275bf0f2508f9026e9394aebc7
There is at least one Gerrit bugfix for an NPE that we should pick up by
this update. There are also improvements to the MINA SSHD server that
gerrit runs.
Full changelogs can be found here:
https://www.gerritcodereview.com/3.8.html#384
Change-Id: Icba387496457c5a60fd914a6ee689104d3a52c1d
Those repos are produced by the Automotive SIG [1], are not used by
OpenStack and increase the size of the centos stream repositories
needlessly.
[1] https://sigs.centos.org/automotive/
Change-Id: I8a12956aa2079ce851ad0bb5ff60f49677f5b7d3
We have successfully removed debian buster from nodepool and zuul at
this point. The last major TODO in debian buster cleanup is to remove it
from our package mirrors. This change is the first step in making that
happen.
For step two we follow the manual process documented in our reprepro
docs [0] for cleaning up mirror components. We will need to perform
these actions against the debian, debian security, and ceph octopus
mirrors.
[0] https://docs.opendev.org/opendev/system-config/latest/reprepro.html#removing-components
Depends-On: https://review.opendev.org/c/openstack/project-config/+/910031
Change-Id: Ic1fc6a45cb7f644d7862312589254b6100e17222
Buster is the old old release of debian having been succeeded by
bullseye and bookworm. Drop buster testing in preparation for buster
test environment removal and add bookworm.
Note the arm64 job is marked nonvoting because there is a bug building
openafs on bookworm. This same issue shows up in nixos [0], and I have
reported it to openafs via their IRC channel where someone is working to
correct the problem upstream. Hopefully we can get a fix backported into
the distro package.
[0] https://github.com/NixOS/nixpkgs/issues/284501
Change-Id: I5b7e2e0cabb5123c48d745e9e84df96130217683
This change updates the opensuse mirror script to stop mirroring
opensuse 15. However, we do not entirely remove the opensuse mirroring
script as it is currently mirring some centos 7 packages from OBS for
kolla. We will clean this up more fully when we remove centos 7.
Depends-On: https://review.opendev.org/c/openstack/project-config/+/909776
Change-Id: I0c3546b79219180b796ca02fa8d82dba2316878a
I have tested this upgrade on a held node going straight from 10.4 to
10.11 in one go. The resulting logs can be found in this paste [0].
The resulting backups of system tables are small enough that it seems
reasonable to keep those enabled (though they can be disabled). Also, we
can either land this change and let docker-compose do the upgrade for
us, or we can put the host in the emergency file, do the upgrade by
hand, then merge this change to reflect the new state of the world.
One advantage to doing this by hand is that we can manually run a db
backup with the service turned off to avoid any lost data between the
time the upgrade occurs and the time of our last backup should anything
go wrong.
In either case we should probably double check that db backups look good
in borg before proceeding. Comments on approach are very much welcome.
[0] https://paste.opendev.org/show/bWhZZH97IMLv44eeiWlB/
Change-Id: I1bfcaeb9b90838a80d002732215f45a14a158fed
Step-by-step process for adding your account to the zuul realm in
Keycloak, so that you can access the admin capabilities of our Zuul
WebUI.
Change-Id: I613e3b45316471df2054300a8b115da78debdcb2
Our deployment tasks wait for Jaeger to be listening on its network
socket, but storage-related delays and slowdowns can sometimes cause
it to take longer than the 120 seconds we budgeted. Increase this to
300 seconds so we can be sure we've given it plenty of time to sort
that out.
Change-Id: I4eaffe2d00fca8b9c10ed9235583fca671413dab
Trivial cleanup of some variable name copy-paste I overlooked,
making the source code for the test clearer.
Change-Id: I5a15e0733b3cf2ceb26f46a2f3d9a9f059d4f702
We should really be backing this up before it begins to get used by
additional services. Also, since our newer deployment uses a
separate RDBMS, back that up safely.
Change-Id: I4510dd05204f4b0f450d1925ed7be148d7d73e6e
The newer Quarkus-based Keycloak container images no longer include
an "auth/" prefix to all the URL paths by default. Rather than alter
the Keycloak deployment, switch Zuul configuration to use the new
default instead.
Change-Id: I9f7f52e80c39c8bd41c728bf9e2b38dcece29978
This is a new server for our Keycloak service. The previous one is
also removed by this change, since it did not have the correct CPU
flags to run the latest Keycloak container images. The problem which
necessitated this rebuild is addressed by an additional check to our
launch script in Ib0f482a939f94e801c82f3583e0a58dc4ca1f35c.
Depends-On: https://review.opendev.org/908608
Change-Id: I4a4a8cb629cbda430a113d61689c9d8ec15408b5
The "UBI" that the latest Keycloak images are based on has a glibc
compiled to only work on x86-64-v2 systems, and in some regions we
seem to sometimes get hypervisors reporting older processor
architectures where it won't work. Check CPU flags for sse4_2
support as an indicator, and abort launching if it's not present.
Change-Id: Ib0f482a939f94e801c82f3583e0a58dc4ca1f35c
