130 Commits

Author SHA1 Message Date
Zuul
2a62ea5c44 Merge "Stop building python3.9 container images" 2023-10-25 14:52:39 +00:00
Zuul
bd3fd30462 Merge "Remove the old mailing list server" 2023-10-20 23:04:26 +00:00
Jeremy Stanley
cab53d10ac Remove the old mailing list server
Clean up references to lists.openstack.org other than as a virtual
host on the new lists01.opendev.org Mailman v3 server. Update a few
stale references to the old openstack-infra mailing list (and
accompanying stale references to the OpenStack Foundation and
OpenStack Infra team). Update our mailing list service documentation
to reflect the new system rather than the old one. Once this change
merges, we can create an archival image of the old server and delete
it (as well as removing it from our emergency skip list for
Ansible).

Side note, the lists.openstack.org server will be 11.5 years old on
November 1, created 2012-05-01 21:14:53 UTC. Farewell, old friend!

Change-Id: I54eddbaaddc7c88bdea8a1dbc88f27108c223239
2023-10-20 18:10:08 +00:00
Clark Boylan
4dc053ff0f Stop building python3.9 container images
Drop the python3.9 container images (python-builder, python-base,
uwsgi-base). At this point nothing is building off of these images and
we should be using python3.11 for anything new.

Note that python3.10 can be cleaned up once zuul-registry and
openstackclient stop building on top of python3.10. Everything else
appears to be python3.11 at this point.

Change-Id: Id30b616ec336a5599766fb808f55e228da686439
2023-10-16 13:52:07 -07:00
Clark Boylan
6e9dfad01b Fix job dependencies on old container images
We recently updated all of these items to build on newer container
images but some of the job dependencies for that were missed. Fix this
before we start cleaning up the old container builds.

Change-Id: I90946007026f9ca84f4fc6f0e5fe5fbf76d21627
2023-10-16 13:50:39 -07:00
James E. Blair
e85ab6f746 Add a debug python base image
This adds a python-base:3.11-bookworm-debug image, which is built
on the normal python:3.11-bookworm upstream image instead of the
slim upstream image.  The normal image includes debug symbols for
the python interpreter which is compiled during its build phase,
so this is the best way to get an opendev python-base image with
debug symbols.

Change-Id: I1d89ac947cd3bea8a468f3ee022fb4cc93bece1f
2023-10-10 08:30:32 -07:00
Clark Boylan
2888408833 Convert haproxy-statsd to bookworm base image
We keep the same python 3.11 version we had on bullseye but switch the
rest of the userspace to bookworm. Since the python version doesn't
change this should have minimal impact.

Change-Id: I59ad8c2a92159f51d567dd0212e2ab8bec1b45b1
2023-09-14 13:36:18 -07:00
Tony Breeds
31c3aa52b9 Update accessbot,ircboot and matrix-eavesdrop to bookworm container
Change-Id: Idb096e21f57df3db1acc3d3afe3881e2e540f965
2023-07-12 14:47:14 +10:00
Tony Breeds
b3fb6641ad Add Debian bookworm based python images
uWSGI: libffi7 has been retired with bullseye so update to libffi8

Change-Id: I3e7e05b0e0148d73e7f0087f456a3f8de10e0c87
2023-07-12 12:18:19 +10:00
Zuul
fd55aa261d Merge "Update accessbot to Use Python 3.11 Base Images" 2023-04-30 23:24:52 +00:00
Gene Kuo
792d3f611f Build haproxy-statsd with Python 3.11 Base Images
This commits update haproxy-statsd Dockerfile to use Python 3.11 images
instead of older 3.9 one. It's part of the effort to update Python
version to 3.10 or later.

Change-Id: I65d469d18d1cf10d7f404b12afdeb5e6d8580bd6
2023-04-27 12:13:54 +09:00
Zuul
f957ebdbb5 Merge "Update Hound to Use Python 3.11 Base Images" 2023-04-27 02:55:13 +00:00
Clark Boylan
276396babc Add Gerrit 3.7 -> 3.8 upgrade job
Now that we have a mostly working 3.8 image it is time to test the
upgrade from 3.7 (what we run in prod) to 3.8 (what we will eventually
run in prod).

Change-Id: Ied8bae6b80cff79668a293ae2f30498abbf6839d
2023-04-26 14:31:37 -07:00
Clark Boylan
40390f996e Add Gerrit 3.8 image builds and test jobs
Gerrit 3.8 isn't officially released yet, but they have relaesed their
third release candidate. Seems like a good time to start building a 3.8
image and testing it.

Change-Id: I11bcd35379cecde135f471a840b03937166b6663
2023-04-26 13:33:14 -07:00
Clark Boylan
cee951e6a9 Remove Gerrit 3.6 image builds and test jobs
We've been running Gerrit 3.7 for some time now and seem very unlikely
to revert at this point. Clean up the Gerrit 3.6 image builds as we
don't need them anymore.

THis change also comments out the 3.6 -> 3.8 upgrade job. Followup
changes will add 3.8 image builds and test the 3.7 -> 3.8 upgrade
process.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/881595
Change-Id: I759b34e48dcede7ffaa66c83da01b81c4fed4b4f
2023-04-26 10:52:54 -07:00
Gene Kuo
9f86500913 Update Hound to Use Python 3.11 Base Images
This commits update hound Dockerfile to use Python 3.11 images instead
of older 3.9 one. It's part of the effort to update Python version to
3.10 or later.

Change-Id: Idf5076709c55d3327c69c6e8160a8348b6640ca7
2023-04-26 19:53:37 +09:00
Gene Kuo
faf0c90fe6 Update accessbot to Use Python 3.11 Base Images
This commits update accessbot Dockerfile to use Python 3.11 images
instead of older 3.9 one. It's part of the effort to update Python
version to 3.10 or later.

Change-Id: I983ca73237b8a11ae3885505f549d8b0b16c144f
2023-04-25 12:29:38 +09:00
Ian Wienand
be992b3bb6
infra-prod: run job against linaro
We have access to manage the linaro cloud, but we don't want to
completely own the host as it has been configured with kolla-ansible;
so we don't want to take over things like name resolution, iptables
rules, docker installation, etc.

But we would like to manage some parts of it, like rolling out our
root users, some cron jobs, etc.  While we could just log in and do
these things, it doesn't feel very openinfra.

This allows us to have a group "unmanaged" that skips the base jobs.
The base playbook is updated to skip these hosts.

For now, we add a cloud-linaro prod job that just does nothing so we
can validate the whole thing.  When it's working, I plan to add a few
things as discussed above.

Change-Id: Ie8de70cbac7ffb9d727a06a349c3d2a3b3aa0b40
2023-03-15 12:00:25 +11:00
Clark Boylan
90440837a7 Add Gerrit 3.6 -> 3.7 upgrade test job
This updates the gerrit upgrade testing job to upgrade from 3.6 to 3.7.

This upgrade requires an offline reindex which is new for us since we've
been on Gerrit 3.x. In order to support this offline reindex requirement
the gerrit role is modified to trigger an offline reindex in the role's
start tasks if the flag to do so is set. I expect this will really only
be used in testing, but it allows us to reuse most everything else in
testing and in production which is nice.

Change-Id: Ibe68176970394cbe71c3126ff3fe7a1b0601b09a
2023-01-13 12:04:47 -08:00
Clark Boylan
34a20bb39c Add Gerrit 3.7 images
This runs system-config-run-review-3.7 but does not add the upgrade test
yet.

Change-Id: I120081b3e30f639a4242fcca7b93def162c240db
2023-01-13 08:40:51 -08:00
Clark Boylan
2cbd400c4e Remove Gerrit 3.5 images
This disables the upgrade testing temporarily with the intention of
adding it back again when we build 3.7 images and can test the 3.6 to
3.7 upgrade.

Change-Id: I102795248ab9e692e4558257d94d5b3e0b73267e
2023-01-10 12:32:32 -08:00
Zuul
b879e5fad7 Merge "Fork the maxking/docker-mailman images" 2022-11-22 18:11:24 +00:00
Zuul
b7b2157133 Merge "Add a mailman3 list server" 2022-11-22 18:00:30 +00:00
Clark Boylan
12d4355385 Fork the maxking/docker-mailman images
These images have a number of issues we've identified and worked
around. The current iteration of this change is essentially
identical to upstream but with a minor tweak to allow the latest
mailman version, and adjusts the paths for hyperkitty and postorius
URLs to match those in the upstream mailman-web codebase, but
doesn't try to address the other items. However, we should consider
moving our fixes from ansible into the docker images where possible
and upstream those updates.

Unfortunately upstream hasn't been super responsive so far hence this
fork. For tracking purposes here are the issues/PRs we've already filed
upstream:

  https://github.com/maxking/docker-mailman/pull/552
  https://github.com/maxking/docker-mailman/issues/548
  https://github.com/maxking/docker-mailman/issues/549
  https://github.com/maxking/docker-mailman/issues/550

Change-Id: I3314037d46c2ef2086a06dea0321d9f8cdd35c73
2022-11-21 16:51:02 +00:00
Clark Boylan
c1c91886b4 Add a mailman3 list server
This should now be a largely functional deployment of mailman 3. There
are still some bits that need testing but we'll use followup changes to
force failure and hold nodes.

This deployment of mailman3 uses upstream docker container images. We
currently hack up uids and gids to accomodate that. We also hack up the
settings file and bind mount it over the upstream file in order to use
host networking. We override the hyperkitty index type to xapian. All
list domains are hosted in a single installation and we use native
vhosting to handle that.

We'll deploy this to a new server and migrate one mailing list domain at
a time. This will allow us to start with lists.opendev.org and test
things like dmarc settings before expanding to the remaining lists.

A migration script is also included, which has seen extensive
testing on held nodes for importing copies of the production data
sets.

Change-Id: Ic9bf5cfaf0b87c100a6ce003a6645010a7b50358
2022-11-11 23:20:19 +00:00
Ian Wienand
730fcf0171 Remove old bridge testing
We don't need to test on a bionic bridge any more, remove the old test
job.

Change-Id: I826f740b004bdc8b85977ba08f4e17e92f40a316
2022-11-03 04:10:31 +00:00
Zuul
b8326dcc9d Merge "Add python 3.11 docker images" 2022-10-27 23:11:29 +00:00
Zuul
39387607bc Merge "Drop python 3.8 base image builds" 2022-10-25 22:06:30 +00:00
Clark Boylan
ee359c7e3b Add python 3.11 docker images
Python 3.11 has been released. Once the parent commit of this commit
lands we will have removed our python3.8 images making room for
python3.11 in our image list. Add these new images which will make way
for running and testing our software on this new version of python.

Change-Id: Idcea3d6fa22839390f63cd1722bc4cb46a6ccd53
2022-10-25 10:43:29 -07:00
Ian Wienand
dc18968927
Run a base test against "old" bridge
Run a base test against a Bionic bridge to ensure we don't break
things as we transition the current production host as we move to a
new Focal-based environment.

Change-Id: I1f745a06c4428cf31a166b3d53dd6321bfd41ebc
2022-10-20 09:49:10 +11:00
Clark Boylan
df4f11393b Drop python 3.8 base image builds
Python 3.11 is coming up and running image builds for all the pythons
gets overwhelming fast. We end up with so many jobs that landing any one
change to our base images becomes difficult. To reduce the total count
we remove builds for 3.8 to make room for 3.11. Only a few things appear
to still be using the 3.8 images and their updates are all listed as
depends on below.

Depends-On: https://review.opendev.org/c/opendev/gerritbot/+/861474
Depends-On: https://review.opendev.org/c/opendev/grafyaml/+/861475
Depends-On: https://review.opendev.org/c/opendev/statusbot/+/861476
Depends-On: https://review.opendev.org/c/zuul/zuul-client/+/861477
Depends-On: https://review.opendev.org/c/zuul/zuul-operator/+/861478
Depends-On: https://review.opendev.org/c/zuul/zuul-registry/+/861479
Change-Id: Ifa44ed0586f54b7ee4d6e37ba32235d63a30addb
2022-10-14 14:34:03 -07:00
James E. Blair
c661fb0972 Add Jaeger tracing server
Change-Id: I1aa68b1d5f99364fa09776301894b922ed169a3a
2022-09-15 19:21:33 -07:00
Ian Wienand
46bb73d947 system-config-run-borg-backup: add to gate
We must have missed this, I noticed when it didn't run on the gate job
for I949c40e9046008d4f442b322a267ce0c967a99dc

Change-Id: I62c5c0f262d9bd53580367dc9f1ad00fe7b6f6f2
2022-08-11 13:54:52 +10:00
Clark Boylan
d40f5d3089 Add Gerrit 3.5 to 3.6 upgrade testing
This adds upgrade testing from our current Gerrit version (3.5) to the
likely future version of our next upgrade (3.6).

To do so we have to refactor the gerrit testing becase the 3.5 to 3.6
upgrade requires we run a command against 3.5. The previous upgrade
system assumed the old version could be left alone and jumped straight
into the upgrade finally testing the end state. Now we have split up the
gerrit bootstrapping and gerrit testing so that normal gerrit testing
and upgrade testing can run these different tasks at different points in
the gerrit deployment process.

Now the upgrade tests use the bootstrapping playbook to create users,
projects, and changes on the old version of gerrit before running the
copy-approvals command. Then after the upgrade we run the test assertion
portion of the job.

Change-Id: Id58b27e6f717f794a8ef7a048eec7fbb3bc52af6
2022-06-22 10:58:17 -07:00
Clark Boylan
1da5615477 Add Gerrit 3.6 jobs
This adds Gerrit 3.6 image build jobs as well as CI testing for this
version of Gerrit. Once we've got images that build and function
generally we'll reenable the upgrade job and work through that.

Change-Id: I494a21911a2279228e57ff8d2b731b06a1573438
2022-06-21 16:54:36 -07:00
Clark Boylan
063ec0f5a7 Remove Gerrit 3.4 jobs
This removes our Gerrit 3.4 image builds as well as testing. We should
land this after an appropriate amount of time has passed since the 3.5
upgrade that we are unlikely to revert.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/847057
Change-Id: Iefa7cc1157311f0239794b15bea7c93f0c625a93
2022-06-21 16:27:46 -07:00
Clark Boylan
cb905c9b4f Fix gerrit deployment dependencies
We've upgraded to Gerrit 3.5 so now need to wait for the 3.5 image to
promote rather than the 3.4 image when deploying Gerrit.

Change-Id: Ic3a4d578aea955aeee51f4cac7f4c95de931a94b
2022-06-21 07:48:12 -07:00
Clark Boylan
90effa2af0 Add Bullseye Python 3.10 base images
Now that we've cleaned up the old unused images we can look forward to
new Python. Add Python 3.10 base images based on Bullseye.

As part of this process we update the default var values in our
Dockerfiles to set Bullseye and Python3.10 as our defaults as these
should be valid for some time. We also tidy up some yaml anchor names to
make future copy and paste for new versions of images easier to perform
text replacement on.

Change-Id: I4943a9178334c4bdf10ee5601e39004d6783b34c
2022-04-20 08:39:52 -07:00
Clark Boylan
2d43f9322b Remove python3.7-bullseye docker images
Everything is running on 3.8 or newer which should allow us to remove
the 3.7 images. This reduces the total set before we add python3.10
images and acts as good cleanup.

Change-Id: I2cc02fd681485f35a1b0bf1c089a12a4c5438df3
2022-04-18 13:42:30 -07:00
Clark Boylan
78999b020e Remove our buster python images
We've moved onto bullseye for just about everything at this point. It is
possible there are stragglers and the removal of these jobs should flag
them if their dependencies and requirements are set properly. Otherwise
they'll continue to pull the historical builds on docker hub. Either way
we'll either shake them out or they will continue until they can move to
bullseye.

We remove these in preparation for adding python3.10 images which don't
make sense for buster and our total image catalog is getting large
enough that successfully building and promoting this entire set is
getting problematic. A bit of spring cleaning on what we can commit to
before we commit to some new stuff.

Depends-On: https://review.opendev.org/c/opendev/gear/+/838402
Depends-On: https://review.opendev.org/c/opendev/storyboard/+/838403
Change-Id: I58c4d314ca4f4be3f1e17ec267a4c324cabf0c2a
2022-04-18 13:42:03 -07:00
Clark Boylan
17b2545807 Fixup some image dependencies in our zuul configs
We don't use buster images anymore for anything. Update our dependency
on buster indicators to up to date and current bullseye dependencies.

Change-Id: I8da237559e074ae3d44be1dde8ffb7da89104d4f
2022-04-18 12:48:39 -07:00
James E. Blair
f196aa047e Clean up some gitea-lb zuul config
This triggers the test job on changes to any gitea.* roles, including
gitea-lb which wasn't included before.

It also removes the letescrypt job as a soft dependency from the lb
jobs since that is not strictly necessary.

Change-Id: Ie5bcd4d8215bb14d939dddf3e20d3173ccc0acdc
2022-02-10 23:38:59 +00:00
James E. Blair
2a9553ef25 Add Zuul load balancer
This adds a load balancer for zuul-web and fingergw.

Change-Id: Id5aa01151f64f3c85e1532ad66999ef9471c5896
2022-02-10 13:24:42 -08:00
Clark Boylan
a5671ddaf3 Update infra-prod-service-review dependencies
We removed the promote jobs for Gerrit 3.3 images but left them in place
as infra-prod-service-review dependencies. Fix that by updating the
infra prod job dependencies to the job for Gerrit 3.4 image promotion.

Change-Id: If2277799db91ea61aaffafb600f403531a0fb562
2022-02-07 13:03:50 -08:00
Clark Boylan
a2dce17612 Test Gerrit upgrade from 3.4 to 3.5
This reenables Gerrit upgrade testing but tests the 3.4 to 3.5 upgrade
now. Note this may need some work to get happy once we have 3.5 images
which is why we've split it out into a separate change.

Change-Id: Ibbbd3f98ac2df8d99d4ffda57df59f4a47da3cd3
2022-02-03 08:11:31 -08:00
Clark Boylan
a3cc983502 Add Gerrit 3.5 image builds and testing
This will build gerrit 3.5 images and run it through our standard Gerrit
testing. Upgrade testing from 3.4 to 3.5 to follow in followup changes.

Change-Id: I76d0389d1455e62b242aad1926b3a09830301801
2022-02-03 08:09:14 -08:00
Zuul
fd3cd75f7c Merge "Stop building Gerrit 3.3 images" 2022-02-03 05:50:59 +00:00
Clark Boylan
4e9da3c255 Stop building Gerrit 3.3 images
We've upgraded to 3.4 and don't appear to be reverting. Remove the 3.3
images as they are no longer needed.

Note we comment out the review upgrade testing jobs until we have 3.5
images building.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/827562
Change-Id: I0e3cb81b790ab06c690ed0245526e4f47911c584
2022-02-02 14:03:28 -08:00
Ian Wienand
545feac217 infra-prod-grafana: drop system-config-promote-image-grafana
We dropped making our own grafana container with with
If0d584f848f213aeea385885e3decfaee6303de5, so we don't need this job
any more.

Change-Id: Ide212f25cda6d25e5cc31b0e8d2a65f3759bafdd
2022-01-31 17:13:40 +11:00
James E. Blair
1492f22faa Use grafyaml container image
Instead of building a local grafana image with grafyaml installed,
use the plain upstream grafana image along with the newly created
separate opendev grafyaml image to run the dashboards.

Depends-On: https://review.opendev.org/780119
Change-Id: If0d584f848f213aeea385885e3decfaee6303de5
2022-01-20 09:25:00 +11:00