18855 Commits

Author SHA1 Message Date
Jeremy Stanley
89d01144a1 Clean up old Mailman v2 roles and vars
Now that we no longer run a Mailman v2 server, we can drop all the
automation we used for deploying and maintaining it.

Change-Id: I522cdbef86d1fe491d446e4b721a7873564c927a
2023-10-31 18:20:12 +00:00
Zuul
ce24cd6a23 Merge "Merge production and test node mailman configs" 2023-10-31 17:03:58 +00:00
Zuul
5b837c1799 Merge "Convert commentlinks to new no html system" 2023-10-30 20:18:38 +00:00
Jeremy Stanley
73f0a5336a Merge production and test node mailman configs
Now that the Mailman v3 migration is complete, we no longer need any
divergence between the lists01 (production) and lists99 (test node)
host vars, so put everything into the group vars file instead.

Change-Id: If92943694e95ef261fbd254eff65a51d8d3f7ce5
2023-10-30 19:26:03 +00:00
Zuul
31a430db07 Merge "Revert "Cap ruamel.yaml install for ARA"" 2023-10-26 22:10:14 +00:00
Zuul
b79818feae Merge "Add OpenInfra EU mailing lists" 2023-10-25 16:42:34 +00:00
Zuul
5d0f944c3e Merge "Update to Ansible 8 on bridge" 2023-10-25 16:25:10 +00:00
Clark Boylan
5aec9da11e Revert "Cap ruamel.yaml install for ARA"
This reverts commit a77eebe911b9651575c32dec8cb5ac84e4057192.

Ruamel.yaml 0.18.2 converted the error assocaited with the use of this
deprecated method from a sys.exit(1) to a raised Exception. It is
believed that this will allow Ara to run in some capacity and we don't
need to pin this dependency anymore.

More details in the upstream bug here:

  https://github.com/ansible-community/ara/issues/524

Change-Id: I694b8a016755d828490f0bcf4c6ceb812edf43d9
2023-10-25 09:04:57 -07:00
Jeremy Stanley
704321653b Add OpenInfra EU mailing lists
The OpenInfra Foundation executive team is requesting creation of
new mailing lists on lists.openinfra.dev for the foundation's new EU
hub. One list will have an open subscription policy and publicly
available archives, while the other will be utilized by the advisory
board for any sensitive topics that must be kept private.

Change-Id: I138bcdddd8b8feeb94adb71f0ba5e03d8c809e20
2023-10-25 15:31:37 +00:00
Zuul
2a62ea5c44 Merge "Stop building python3.9 container images" 2023-10-25 14:52:39 +00:00
Zuul
a045d7590a Merge "Add zk test to check myid is set in service" 2023-10-24 18:38:03 +00:00
Clark Boylan
a77eebe911 Cap ruamel.yaml install for ARA
ARA is not compatible with latest ruamel.yaml which leads to errors
running ansible. Fix this by capping the ruamel.yaml version we install.

Change-Id: Ia5db3ba8579e7e5c1fe375b156323b94f341ad3e
2023-10-24 09:44:16 -07:00
Clark Boylan
8f9b1f2c9c Convert commentlinks to new no html system
Gerrit 3.8 drops support for html in commentlinks entirely. Gerrit 3.7
supports both html and the new non html system. Update our 3.7
installation to the new system on 3.7 so that we are ready for the
Gerrit 3.8 upgrade later.

Most of our comment links did not use html entries so we drop the html
lines entirely. A single commentlink does use html and there we convert
it to the new prefix, link, text, suffix system. More details can be
found here:

  https://gerrit.googlesource.com/gerrit/+/refs/tags/v3.8.2/tools/migration/html_to_link_commentlink.md

This should be a 1:1 mapping for our config and not change any behavior.

Change-Id: I0b87aac7b90814d242338be8fd03cfc9a76200f7
2023-10-23 14:03:06 +00:00
Zuul
bd3fd30462 Merge "Remove the old mailing list server" 2023-10-20 23:04:26 +00:00
Jeremy Stanley
cab53d10ac Remove the old mailing list server
Clean up references to lists.openstack.org other than as a virtual
host on the new lists01.opendev.org Mailman v3 server. Update a few
stale references to the old openstack-infra mailing list (and
accompanying stale references to the OpenStack Foundation and
OpenStack Infra team). Update our mailing list service documentation
to reflect the new system rather than the old one. Once this change
merges, we can create an archival image of the old server and delete
it (as well as removing it from our emergency skip list for
Ansible).

Side note, the lists.openstack.org server will be 11.5 years old on
November 1, created 2012-05-01 21:14:53 UTC. Farewell, old friend!

Change-Id: I54eddbaaddc7c88bdea8a1dbc88f27108c223239
2023-10-20 18:10:08 +00:00
Zuul
69eaeeab88 Merge "Fix job dependencies on old container images" 2023-10-18 17:43:14 +00:00
Clark Boylan
27e65ba84f Update to Ansible 8 on bridge
Zuul has already made the move; we should catch up. Part of this is
motivated by the weird failures we've seen when creating the LE
certcheck domains list in an Ansible loop though I've no real evidence
that upgrading would fix this. Python on bridge is 3.10 which should be
compatible with Ansible 8.

Full (and probably far too dense) changelogs can be found here:

  https://github.com/ansible-community/ansible-build-data/blob/main/8/CHANGELOG-v8.rst

A prior patchset temporarily updated zuul configs to run most of our
system-config-run-* jobs using ansible 8. They all passed implying that
our playbooks and roles will function under the newer version of
ansible.

Change-Id: Ie1b4e5363c56c0dcd61721fb0ea061d5198ecfed
2023-10-18 09:20:31 -07:00
Clark Boylan
4929d84f49 Fix the linaro cloud certcheck entry
The entry we had did not match the name used by nodepool. This caused us
to not detect that this cert would expire (and has now expired).

Change-Id: Ibd4885f2f3fbbc776f76fef92736b98b8f87c664
2023-10-18 08:43:33 -07:00
Zuul
0218fe84ed Merge "Bump zookeeper from 3.7 to 3.8" 2023-10-17 17:10:00 +00:00
Clark Boylan
5bbbccee18 Add zk test to check myid is set in service
Previously we were checking that myid was written to disk in the
expected location. However, it is possible that zk would stop looking in
that location for the myid value. To ensure we actually set the value in
the running service check the logs for the [myid:4] string.

Change-Id: Iee3b126abac13e19dab9ddf4c64ed133d0a98956
2023-10-17 09:31:16 -07:00
Clark Boylan
3a4ce1bd08 Fix the Ansible Galaxy proxy testinfra test
Previously we checked that "Ansible Galaxy" shows up in the html result
requesting the root of the Galaxy proxy. This now fails and looking at
the results of the fetch the title of the page is "Galaxy NG". Update
our test to check for "Galaxy NG" instead.

Additionally our content checks of actual collections are affected by an
api bump from v2 to v3. Among other things this appears to be a
completely new implementation that does not have backward compatible
support for v2 and may require authentication to use. I've commented out
our old test content for the content checks and someone will need to fix
this later.

Change-Id: I6b17eea82ac95200ba5069de74e9a7dc30d6fed8
2023-10-16 15:26:21 -07:00
Clark Boylan
4dc053ff0f Stop building python3.9 container images
Drop the python3.9 container images (python-builder, python-base,
uwsgi-base). At this point nothing is building off of these images and
we should be using python3.11 for anything new.

Note that python3.10 can be cleaned up once zuul-registry and
openstackclient stop building on top of python3.10. Everything else
appears to be python3.11 at this point.

Change-Id: Id30b616ec336a5599766fb808f55e228da686439
2023-10-16 13:52:07 -07:00
Clark Boylan
6e9dfad01b Fix job dependencies on old container images
We recently updated all of these items to build on newer container
images but some of the job dependencies for that were missed. Fix this
before we start cleaning up the old container builds.

Change-Id: I90946007026f9ca84f4fc6f0e5fe5fbf76d21627
2023-10-16 13:50:39 -07:00
Clark Boylan
53fe07271c Noop change to retrigger lists3 deployment
We've been trying to get this to deploy automatically without much
success due to a couple of unrelated errors. The most recent appears to
possibly be an ansible issue within ansible itself (eg not our
playbooks). Land a noop change to retrigger things and see if this is
consistent or not.

Change-Id: Iaf0aa14a82fb7d0a2b61a5138c7435d3eda21a3e
2023-10-16 11:54:48 -07:00
Clark Boylan
944b78154d Fix the relevant files lists for lists3 jobs
Fix the infra-prod-service-lists3 job to trigger when we update the
mailman3.yaml group vars file. In addition we make a noop reorganization
change to the mailman3 group file to group exim vars together which will
be used to ensure that this change triggers the lists3 job as expected.

In system-config-run-lists3 we update that job to be triggered when we
update the docker images for mailman. We don't bother testing this now
as that would be masked off by the update to the mailman3 groups file.
But in the future when we do mailman3 image updates we'll be looking for
this job to run.

Change-Id: I994b0a79bf46f525dd9e059719f5a08c9c390b8c
2023-10-15 19:52:01 -07:00
Jeremy Stanley
82b5640ff4 Drop the mailman_copy Exim router
In Ic1156849957bc326e9216c2aca0ab9d180e158e6 we added a temporary
router named mailman_copy to dump raw messages for the
openstack-discuss mailing list to an mbox file at
/var/mail/openstack-discuss in order to be able to compare
pre-Mailman state of messages for DKIM signature debugging. Since
this file doesn't exist and Exim lacks permission to create it, the
resulting router errors are leading to message deferrals for the
openstack-discuss mailing list.

Rather than add Ansible to create the mbox file for this, just drop
the router and accompanying transport definitions from our Exim
config. We can always set it up more thoroughly in the future if we
ever want to re-add it.

Change-Id: If4f6c7b90b7b312b23a7736251f704dace668879
2023-10-15 01:04:47 +00:00
Jeremy Stanley
8991d4b160 Add Mailman v3 server to Cacti
This was missed when the server was initially brought up.

Change-Id: I23dda2ed4ffa6547104f4769390bfbc51ce34110
2023-10-12 19:44:10 +00:00
Zuul
3d8fab4ff6 Merge "Remove ara from source install option" 2023-10-12 18:07:39 +00:00
Zuul
e772eaced7 Merge "Move OpenStack lists to Mailman 3" 2023-10-12 13:14:04 +00:00
Zuul
51003247f8 Merge "Update Gerrit to 3.7.5" 2023-10-11 23:31:19 +00:00
Jeremy Stanley
f4902e98fd Move OpenStack lists to Mailman 3
This uncomments the list additions for the lists.openstack.org site
on the new mailman server, and should be merged the day of the
maintenance prior to the start of the scheduled outage window.

Separately removing configuration from the old server is
unnecessary, as there will be a cleanup change merged after the
maintenance window to remove all files associated with it and clear
it out of our inventory in preparation for archival imaging and
deletion (the old server will have its services disabled and be kept
in our emergency skip list for Ansible until that happens).

Change-Id: I1f6d3c8dfcb2bb98fa5b93bcc2f4a13927c55047
2023-10-11 18:02:46 +00:00
Clark Boylan
93f423b3be Bump zookeeper from 3.7 to 3.8
Note that we'll probably end up manually performing this upgrade to
ensure the leader is upgraded last as well as performing sanity checks
along the way. Please don't merge this change until we are certain we
are ready for it. In the meaintime it gives us early feedback for any
unexpected problems with the new zookeeper version.

Change-Id: I84c8f3d05edba03cd4ab526ab0105d7512e3984f
2023-10-11 08:56:18 -07:00
Clark Boylan
9f024b5fea Rebuild gitea 1.20.5 on latest golang 1.20.10
Upstream golang updates are worth recompiling gitea under. Details can
be found in the golang 1.20 release notes:

  https://go.dev/doc/devel/release#go1.20.minor

Change-Id: I6ddeaa23d5aee23928d6f448095bb69fe82d94a9
2023-10-11 08:53:53 -07:00
James E. Blair
e85ab6f746 Add a debug python base image
This adds a python-base:3.11-bookworm-debug image, which is built
on the normal python:3.11-bookworm upstream image instead of the
slim upstream image.  The normal image includes debug symbols for
the python interpreter which is compiled during its build phase,
so this is the best way to get an opendev python-base image with
debug symbols.

Change-Id: I1d89ac947cd3bea8a468f3ee022fb4cc93bece1f
2023-10-10 08:30:32 -07:00
Zuul
cac37a7a3c Merge "Update gerrit image to bookworm" 2023-10-09 16:19:47 +00:00
Takashi Kajinami
9d89477326 reprepro: mirror Ubuntu UCA Bobcat for Ubuntu Jammy
Change-Id: Iaef326ad9218808e1b2a8e47c31a10f45d06f7b6
2023-10-08 13:46:35 +09:00
Zuul
16744d8336 Merge "Blackhole deliveries for Mailman v3 local user" 2023-10-06 16:56:39 +00:00
Zuul
83f5e33cbc Merge "Update the apache ua filter set" 2023-10-06 16:56:37 +00:00
Zuul
8a9e9ffe1b Merge "Upgrade to gitea 1.20.5" 2023-10-06 16:54:55 +00:00
Clark Boylan
9beab7723b Update the apache ua filter set
There are always more UAs to add to the list. It does look like they are
now using all the chrome versions with android. To match these I've
added regexes to the very end of the list so that we don't have to list
hundreds of rules separately. We might be able to collapse of the
earlier rules too, but in theory having direct matches first is faster?

Change-Id: Ic0a84aabea4327d40fa89aef6eb9f9a2d42a658f
2023-10-05 16:01:05 -07:00
Zuul
40bdcd848c Merge "Add Element X Ignition support for Matrix" 2023-10-05 21:44:50 +00:00
Clark Boylan
f1cc7d4f8e Update gitea base OS during image builds
This ensures we're always up to date with our packages even if the
upstream container images lag behind debian proper. Useful for pulling
in bugfixes more quickly than upstream seems to think we want them.

Change-Id: Ia7ec97ca17ad1175c8ddd4c5d037f516dcdd891a
2023-10-05 10:59:24 -07:00
Zuul
32156d8eb4 Merge "Add mailing list for Nordix environment" 2023-10-05 15:22:10 +00:00
Clark Boylan
dd48296edd Fix python-builder container image system updates
There was a small issue in the recent change to perform system updates
when building the python-builder and python-base images. I didn't
realize that python-builder is a two stage build and we need to do the
update in both stages.

Ultimately this has minimal impact on the final images we produce as
those are all build on python-base not python-builder. But to ensure
some difference during python wheel build time on builder doesn't affect
the install location on the base image we should keep these in sync.

Change-Id: I16159fbb490b0ec2e179381a50b9570c9aacd18f
2023-10-04 08:32:34 -07:00
Clark Boylan
e81e37ad43 Update our base container images
There are new pythons and other things we should update to pull in.

Change-Id: I5430e4b6e17861049d2dd60e88bf330595388e23
2023-10-03 17:04:50 -07:00
Clark Boylan
267e0cb6f0 Upgrade to gitea 1.20.5
This looks like a straightforward bugfix release according to the
release notes [0]. There are also no template changes in the three
templates we override.

[0] https://github.com/go-gitea/gitea/blob/v1.20.5/CHANGELOG.md

Change-Id: Id5521289daeb974ac1ec73ffb85d5adb5780fae8
2023-10-03 09:39:23 -07:00
Jeremy Stanley
fcef589bdc Add mailing list for Nordix environment
As part of the transition from the Nordix group to OpenInfra Europe,
some systems and services will remain under the Nordix name for now.
The people managing these resources need a mailing list to better
coordinate their activities.

Change-Id: I03b679b4d5f57b1953e1815555b79caf5b6452ff
2023-10-03 14:36:32 +00:00
Jeremy Stanley
222414b585 Blackhole deliveries for Mailman v3 local user
On Mailman v2 the "mailman" addresses were mapped to special mailing
lists used for monthly password notifications and some other tasks.
This does not exist on Mailman v3, but spammers still have the old
mailman list addresses and send junk to them, which the server
attempts to deliver because there's a local user account with the
same name.

Reject messages for the old "mailman" addresses at receipt, so they
never enter our message queue.

Change-Id: I9db93ae98f4b3952400c1e478612ab70a6241dd1
2023-10-02 21:33:11 +00:00
Jeremy Stanley
f8a528cf8a Add Element X Ignition support for Matrix
We received a notification from Element Matrix Services recommending
we make this change in order to support the new Element X mobile
client. More info:

https://element.io/blog/element-x-ignition/

Change-Id: Ieaa3bc7c5c6e397de4c1a63a2b67a150dff1f8e2
2023-09-27 13:15:06 +00:00
Zuul
9f953277e3 Merge "Add codesearch to cacti" 2023-09-22 12:12:20 +00:00