18865 Commits

Author SHA1 Message Date
Zuul
a93ad36865 Merge "Upgrade Etherpad to 1.9.4" 2023-11-02 17:34:14 +00:00
Zuul
bd844f01fb Merge "Update Etherpad settings from upstream" 2023-11-02 16:51:15 +00:00
Zuul
014ef1be7a Merge "Upgrade to latest Mailman 3 releases" 2023-11-02 15:51:25 +00:00
Zuul
f1c7d7a8de Merge "Add a jammy test node for regional mirrors" 2023-11-01 22:21:40 +00:00
Zuul
fef38e13aa Merge "[testinfra] Add port into curl's --resolve arg." 2023-11-01 21:28:41 +00:00
Jeremy Stanley
b5d32d39cd Upgrade Etherpad to 1.9.4
The changelogs can be found here:

https://github.com/ether/etherpad-lite/blob/v1.9.3/CHANGELOG.md#193
https://github.com/ether/etherpad-lite/blob/v1.9.4/CHANGELOG.md#194

There doesn't appear to be anything relevant to our deployment in
these updates other than bug fixes and library version increases,
but upgrading now will reduce future deltas.

Change-Id: Ic1629bf8cb140c33a641e1c613d43e8a9d4d0f1e
2023-11-01 18:53:42 +00:00
Jeremy Stanley
09d89298e3 Update Etherpad settings from upstream
Refresh our versions of settings.json.docker and
settings.json.template from upstream, incorporating our local
preferences as edits to the latter (the former is included in the
container image we publish but the latter gets mapped over it during
deployment).

Changes to the required version of node-log4js in Etherpad 1.9.4
will invalidate our custom logging configuration and error out,
preventing the service from starting, so go ahead and remove it now.

Change-Id: Ic05ed9be7b6900ba9cdfa09b28600bcd55b770fd
2023-11-01 18:49:05 +00:00
Tony Breeds
f223a237a2 Add a jammy test node for regional mirrors
Change-Id: I922af92e523407b7324f020732fad52b98f027e1
2023-10-31 18:27:59 -05:00
Tony Breeds
92d2bb7f6b [testinfra] Add port into curl's --resolve arg.
The curl manpage explains that port isn't optional:

 --resolve <[+]host:port:addr[,addr]...>
   Provide a custom address for a specific host and port pair.  Us‐
   ing  this, you can make the curl requests(s) use a specified ad‐
   dress and prevent the otherwise normally resolved address to  be
   used.  Consider  it a sort of /etc/hosts alternative provided on
   the command line. The port number should be the number used  for
   the  specific  protocol  the host will be used for. It means you
   need several entries if you want to provide address for the same
   host but different ports.

Change-Id: I40117768bbc149678a69905a8f6ecd3519301ce1
2023-10-31 17:14:37 -05:00
Jeremy Stanley
89d01144a1 Clean up old Mailman v2 roles and vars
Now that we no longer run a Mailman v2 server, we can drop all the
automation we used for deploying and maintaining it.

Change-Id: I522cdbef86d1fe491d446e4b721a7873564c927a
2023-10-31 18:20:12 +00:00
Zuul
ce24cd6a23 Merge "Merge production and test node mailman configs" 2023-10-31 17:03:58 +00:00
Zuul
5b837c1799 Merge "Convert commentlinks to new no html system" 2023-10-30 20:18:38 +00:00
Jeremy Stanley
73f0a5336a Merge production and test node mailman configs
Now that the Mailman v3 migration is complete, we no longer need any
divergence between the lists01 (production) and lists99 (test node)
host vars, so put everything into the group vars file instead.

Change-Id: If92943694e95ef261fbd254eff65a51d8d3f7ce5
2023-10-30 19:26:03 +00:00
Jeremy Stanley
b312e15b57 Upgrade to latest Mailman 3 releases
New releases info:

https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/4U5AP7GZ76NYQONACUVPDHSJBLLBSENL/

Sync all our forked files from mailman-docker to the current
upstream state, except for our overridden hyperkitty->archives and
postorius->mailman3 URL routes in
docker/mailman/web/mailman-web/urls.py.

Change-Id: I3b3955c8b2b91f167510c8a1122d9d8e2d620082
2023-10-29 16:28:43 +00:00
Zuul
31a430db07 Merge "Revert "Cap ruamel.yaml install for ARA"" 2023-10-26 22:10:14 +00:00
Zuul
b79818feae Merge "Add OpenInfra EU mailing lists" 2023-10-25 16:42:34 +00:00
Zuul
5d0f944c3e Merge "Update to Ansible 8 on bridge" 2023-10-25 16:25:10 +00:00
Clark Boylan
5aec9da11e Revert "Cap ruamel.yaml install for ARA"
This reverts commit a77eebe911b9651575c32dec8cb5ac84e4057192.

Ruamel.yaml 0.18.2 converted the error assocaited with the use of this
deprecated method from a sys.exit(1) to a raised Exception. It is
believed that this will allow Ara to run in some capacity and we don't
need to pin this dependency anymore.

More details in the upstream bug here:

  https://github.com/ansible-community/ara/issues/524

Change-Id: I694b8a016755d828490f0bcf4c6ceb812edf43d9
2023-10-25 09:04:57 -07:00
Jeremy Stanley
704321653b Add OpenInfra EU mailing lists
The OpenInfra Foundation executive team is requesting creation of
new mailing lists on lists.openinfra.dev for the foundation's new EU
hub. One list will have an open subscription policy and publicly
available archives, while the other will be utilized by the advisory
board for any sensitive topics that must be kept private.

Change-Id: I138bcdddd8b8feeb94adb71f0ba5e03d8c809e20
2023-10-25 15:31:37 +00:00
Zuul
2a62ea5c44 Merge "Stop building python3.9 container images" 2023-10-25 14:52:39 +00:00
Zuul
a045d7590a Merge "Add zk test to check myid is set in service" 2023-10-24 18:38:03 +00:00
Clark Boylan
a77eebe911 Cap ruamel.yaml install for ARA
ARA is not compatible with latest ruamel.yaml which leads to errors
running ansible. Fix this by capping the ruamel.yaml version we install.

Change-Id: Ia5db3ba8579e7e5c1fe375b156323b94f341ad3e
2023-10-24 09:44:16 -07:00
Clark Boylan
8f9b1f2c9c Convert commentlinks to new no html system
Gerrit 3.8 drops support for html in commentlinks entirely. Gerrit 3.7
supports both html and the new non html system. Update our 3.7
installation to the new system on 3.7 so that we are ready for the
Gerrit 3.8 upgrade later.

Most of our comment links did not use html entries so we drop the html
lines entirely. A single commentlink does use html and there we convert
it to the new prefix, link, text, suffix system. More details can be
found here:

  https://gerrit.googlesource.com/gerrit/+/refs/tags/v3.8.2/tools/migration/html_to_link_commentlink.md

This should be a 1:1 mapping for our config and not change any behavior.

Change-Id: I0b87aac7b90814d242338be8fd03cfc9a76200f7
2023-10-23 14:03:06 +00:00
Zuul
bd3fd30462 Merge "Remove the old mailing list server" 2023-10-20 23:04:26 +00:00
Jeremy Stanley
cab53d10ac Remove the old mailing list server
Clean up references to lists.openstack.org other than as a virtual
host on the new lists01.opendev.org Mailman v3 server. Update a few
stale references to the old openstack-infra mailing list (and
accompanying stale references to the OpenStack Foundation and
OpenStack Infra team). Update our mailing list service documentation
to reflect the new system rather than the old one. Once this change
merges, we can create an archival image of the old server and delete
it (as well as removing it from our emergency skip list for
Ansible).

Side note, the lists.openstack.org server will be 11.5 years old on
November 1, created 2012-05-01 21:14:53 UTC. Farewell, old friend!

Change-Id: I54eddbaaddc7c88bdea8a1dbc88f27108c223239
2023-10-20 18:10:08 +00:00
Zuul
69eaeeab88 Merge "Fix job dependencies on old container images" 2023-10-18 17:43:14 +00:00
Clark Boylan
27e65ba84f Update to Ansible 8 on bridge
Zuul has already made the move; we should catch up. Part of this is
motivated by the weird failures we've seen when creating the LE
certcheck domains list in an Ansible loop though I've no real evidence
that upgrading would fix this. Python on bridge is 3.10 which should be
compatible with Ansible 8.

Full (and probably far too dense) changelogs can be found here:

  https://github.com/ansible-community/ansible-build-data/blob/main/8/CHANGELOG-v8.rst

A prior patchset temporarily updated zuul configs to run most of our
system-config-run-* jobs using ansible 8. They all passed implying that
our playbooks and roles will function under the newer version of
ansible.

Change-Id: Ie1b4e5363c56c0dcd61721fb0ea061d5198ecfed
2023-10-18 09:20:31 -07:00
Clark Boylan
4929d84f49 Fix the linaro cloud certcheck entry
The entry we had did not match the name used by nodepool. This caused us
to not detect that this cert would expire (and has now expired).

Change-Id: Ibd4885f2f3fbbc776f76fef92736b98b8f87c664
2023-10-18 08:43:33 -07:00
Zuul
0218fe84ed Merge "Bump zookeeper from 3.7 to 3.8" 2023-10-17 17:10:00 +00:00
Clark Boylan
5bbbccee18 Add zk test to check myid is set in service
Previously we were checking that myid was written to disk in the
expected location. However, it is possible that zk would stop looking in
that location for the myid value. To ensure we actually set the value in
the running service check the logs for the [myid:4] string.

Change-Id: Iee3b126abac13e19dab9ddf4c64ed133d0a98956
2023-10-17 09:31:16 -07:00
Clark Boylan
3a4ce1bd08 Fix the Ansible Galaxy proxy testinfra test
Previously we checked that "Ansible Galaxy" shows up in the html result
requesting the root of the Galaxy proxy. This now fails and looking at
the results of the fetch the title of the page is "Galaxy NG". Update
our test to check for "Galaxy NG" instead.

Additionally our content checks of actual collections are affected by an
api bump from v2 to v3. Among other things this appears to be a
completely new implementation that does not have backward compatible
support for v2 and may require authentication to use. I've commented out
our old test content for the content checks and someone will need to fix
this later.

Change-Id: I6b17eea82ac95200ba5069de74e9a7dc30d6fed8
2023-10-16 15:26:21 -07:00
Clark Boylan
4dc053ff0f Stop building python3.9 container images
Drop the python3.9 container images (python-builder, python-base,
uwsgi-base). At this point nothing is building off of these images and
we should be using python3.11 for anything new.

Note that python3.10 can be cleaned up once zuul-registry and
openstackclient stop building on top of python3.10. Everything else
appears to be python3.11 at this point.

Change-Id: Id30b616ec336a5599766fb808f55e228da686439
2023-10-16 13:52:07 -07:00
Clark Boylan
6e9dfad01b Fix job dependencies on old container images
We recently updated all of these items to build on newer container
images but some of the job dependencies for that were missed. Fix this
before we start cleaning up the old container builds.

Change-Id: I90946007026f9ca84f4fc6f0e5fe5fbf76d21627
2023-10-16 13:50:39 -07:00
Clark Boylan
53fe07271c Noop change to retrigger lists3 deployment
We've been trying to get this to deploy automatically without much
success due to a couple of unrelated errors. The most recent appears to
possibly be an ansible issue within ansible itself (eg not our
playbooks). Land a noop change to retrigger things and see if this is
consistent or not.

Change-Id: Iaf0aa14a82fb7d0a2b61a5138c7435d3eda21a3e
2023-10-16 11:54:48 -07:00
Clark Boylan
944b78154d Fix the relevant files lists for lists3 jobs
Fix the infra-prod-service-lists3 job to trigger when we update the
mailman3.yaml group vars file. In addition we make a noop reorganization
change to the mailman3 group file to group exim vars together which will
be used to ensure that this change triggers the lists3 job as expected.

In system-config-run-lists3 we update that job to be triggered when we
update the docker images for mailman. We don't bother testing this now
as that would be masked off by the update to the mailman3 groups file.
But in the future when we do mailman3 image updates we'll be looking for
this job to run.

Change-Id: I994b0a79bf46f525dd9e059719f5a08c9c390b8c
2023-10-15 19:52:01 -07:00
Jeremy Stanley
82b5640ff4 Drop the mailman_copy Exim router
In Ic1156849957bc326e9216c2aca0ab9d180e158e6 we added a temporary
router named mailman_copy to dump raw messages for the
openstack-discuss mailing list to an mbox file at
/var/mail/openstack-discuss in order to be able to compare
pre-Mailman state of messages for DKIM signature debugging. Since
this file doesn't exist and Exim lacks permission to create it, the
resulting router errors are leading to message deferrals for the
openstack-discuss mailing list.

Rather than add Ansible to create the mbox file for this, just drop
the router and accompanying transport definitions from our Exim
config. We can always set it up more thoroughly in the future if we
ever want to re-add it.

Change-Id: If4f6c7b90b7b312b23a7736251f704dace668879
2023-10-15 01:04:47 +00:00
Jeremy Stanley
8991d4b160 Add Mailman v3 server to Cacti
This was missed when the server was initially brought up.

Change-Id: I23dda2ed4ffa6547104f4769390bfbc51ce34110
2023-10-12 19:44:10 +00:00
Zuul
3d8fab4ff6 Merge "Remove ara from source install option" 2023-10-12 18:07:39 +00:00
Zuul
e772eaced7 Merge "Move OpenStack lists to Mailman 3" 2023-10-12 13:14:04 +00:00
Zuul
51003247f8 Merge "Update Gerrit to 3.7.5" 2023-10-11 23:31:19 +00:00
Jeremy Stanley
f4902e98fd Move OpenStack lists to Mailman 3
This uncomments the list additions for the lists.openstack.org site
on the new mailman server, and should be merged the day of the
maintenance prior to the start of the scheduled outage window.

Separately removing configuration from the old server is
unnecessary, as there will be a cleanup change merged after the
maintenance window to remove all files associated with it and clear
it out of our inventory in preparation for archival imaging and
deletion (the old server will have its services disabled and be kept
in our emergency skip list for Ansible until that happens).

Change-Id: I1f6d3c8dfcb2bb98fa5b93bcc2f4a13927c55047
2023-10-11 18:02:46 +00:00
Clark Boylan
93f423b3be Bump zookeeper from 3.7 to 3.8
Note that we'll probably end up manually performing this upgrade to
ensure the leader is upgraded last as well as performing sanity checks
along the way. Please don't merge this change until we are certain we
are ready for it. In the meaintime it gives us early feedback for any
unexpected problems with the new zookeeper version.

Change-Id: I84c8f3d05edba03cd4ab526ab0105d7512e3984f
2023-10-11 08:56:18 -07:00
Clark Boylan
9f024b5fea Rebuild gitea 1.20.5 on latest golang 1.20.10
Upstream golang updates are worth recompiling gitea under. Details can
be found in the golang 1.20 release notes:

  https://go.dev/doc/devel/release#go1.20.minor

Change-Id: I6ddeaa23d5aee23928d6f448095bb69fe82d94a9
2023-10-11 08:53:53 -07:00
James E. Blair
e85ab6f746 Add a debug python base image
This adds a python-base:3.11-bookworm-debug image, which is built
on the normal python:3.11-bookworm upstream image instead of the
slim upstream image.  The normal image includes debug symbols for
the python interpreter which is compiled during its build phase,
so this is the best way to get an opendev python-base image with
debug symbols.

Change-Id: I1d89ac947cd3bea8a468f3ee022fb4cc93bece1f
2023-10-10 08:30:32 -07:00
Zuul
cac37a7a3c Merge "Update gerrit image to bookworm" 2023-10-09 16:19:47 +00:00
Takashi Kajinami
9d89477326 reprepro: mirror Ubuntu UCA Bobcat for Ubuntu Jammy
Change-Id: Iaef326ad9218808e1b2a8e47c31a10f45d06f7b6
2023-10-08 13:46:35 +09:00
Zuul
16744d8336 Merge "Blackhole deliveries for Mailman v3 local user" 2023-10-06 16:56:39 +00:00
Zuul
83f5e33cbc Merge "Update the apache ua filter set" 2023-10-06 16:56:37 +00:00
Zuul
8a9e9ffe1b Merge "Upgrade to gitea 1.20.5" 2023-10-06 16:54:55 +00:00
Clark Boylan
9beab7723b Update the apache ua filter set
There are always more UAs to add to the list. It does look like they are
now using all the chrome versions with android. To match these I've
added regexes to the very end of the list so that we don't have to list
hundreds of rules separately. We might be able to collapse of the
earlier rules too, but in theory having direct matches first is faster?

Change-Id: Ic0a84aabea4327d40fa89aef6eb9f9a2d42a658f
2023-10-05 16:01:05 -07:00