19211 Commits

Author SHA1 Message Date
James E. Blair
b255615c99 Configure zuul-launcher to use its logging config file
This file exists, but is not referenced in zuul.conf.

Change-Id: I8244dcffd43697a03b0f559ac63e2a185fb35537
2024-10-09 15:01:37 -07:00
James E. Blair
f3f3d8adee Remove zuul_ssh_private_key_contents from scheduler host vars
This is defined in the big zuul group (and used by the zuul role),
 so let's just rely on that definition.

Change-Id: I2c5b5d744a3506717d9c4ad15fa04344bb5890bd
2024-10-09 12:45:16 -07:00
Zuul
49d4eed605 Merge "Update Mailman containers to latest versions" 2024-10-09 19:36:58 +00:00
Zuul
550508440f Merge "Add an inventory entry for our first Zuul launcher" 2024-10-09 17:59:59 +00:00
Zuul
0e8bd76019 Merge "Drop more Xenial testing from system-config" 2024-10-09 17:35:19 +00:00
Jeremy Stanley
bd092b9552 Add an inventory entry for our first Zuul launcher
This adds zl01.opendev.org to the Ansible inventory, as an initial
Zuul launcher server.

Depends-On: https://review.opendev.org/931917
Depends-On: https://review.opendev.org/922680
Change-Id: I561a9873bc5a4636f8d717727816cb342b6d7aea
2024-10-09 17:09:46 +00:00
Zuul
8fc9da4578 Merge "Explicitly down Jitsi-Meet containers on upgrade" 2024-10-04 18:52:33 +00:00
Zuul
60b3e322ff Merge "reprepro: mirror Ubuntu UCA Dalmatian for Ubuntu Noble" 2024-10-04 14:19:33 +00:00
Jeremy Stanley
24acb98c8d Update Mailman containers to latest versions
This upgrades our images to Alpine 3.20, Django 4.2, Mailman 3.3.10,
Postorius 1.3.13, Hyperkitty 1.3.12 and django-mailman3 1.3.15.
Files are re-synced with upstream (either container or project)
files, with versions and any alterations noted.

Change-Id: I78d37c0635d38ecfc1d1143a69892fe8d8685214
2024-10-02 16:37:08 +00:00
Jeremy Stanley
4f0bed6998 Explicitly down Jitsi-Meet containers on upgrade
We've been running into problems with Jitsi-Meet processes not
restarting cleanly if we only docker-compose up after image pulls.
Instead, check for whether the pull resulted in new image downloads
and then cleanly down and up the containers, which seems to address
the problem based on manual testing on the servers.

Change-Id: Id551767e72102a4b8667aa9dacc81755f332a278
2024-09-26 17:52:29 +00:00
Zuul
ab28acf052 Merge "Update Gitea to v1.22.2" 2024-09-25 19:27:05 +00:00
Jonathan Rosser
e701af7716 reprepro: mirror Ubuntu UCA Dalmatian for Ubuntu Noble
Change-Id: I3a692457af110c00f979df8c7c7288f012899539
2024-09-24 12:47:52 +00:00
Clark Boylan
1fd4deb76d Update Etherpad to v2.2.5
After the 2.2.4 release we upgraded to a development commit between
2.2.4 and the future (at that time non existant) 2.2.5 release to fix
integration between meetpad and etherpad. Now there is a proper 2.2.5
and we should update to get off the dev commit.

This release fixes a number of bugs, updates dependencies, and adds
proper swagger documentation for the API. The "complete" changelog can
be seen here:
  https://github.com/ether/etherpad-lite/blob/v2.2.5/CHANGELOG.md

Note that I don't believe the API has changed they are merely
documenting it properly using swagger. Our testing should confirm.
To expose the new swagger documentation we do add /api-docs/ and
/api-docs.json to our proxy exclusion list.

We also update our settings.json files to sync with upstream. This pulls
in a new `updateServer` key value pair to set the location that should
be checked by etherpad to determine if there is a newer version
available. I believe this behavior has existing for years they are just
now making it a bit more configurable. Unfortunately the way this value
is used I think we will do a local file lookup if we set the value to
"". I've stuck with the default since this shouldn't be a regression and
we can try to disable it later.

Change-Id: I73a09a0c79db18887cb1703c84f9aebae6f072eb
2024-09-23 10:29:15 -07:00
Clark Boylan
ddbfa8b162 Update Gitea to v1.22.2
We're running v1.22.1 currently and v1.22.2 fixes a number of bugs. The
full changelog can be seen here:
  https://github.com/go-gitea/gitea/blob/v1.22.2/CHANGELOG.md

The template files we override have not changed and neither has the
Dockerfile since we last upgraded. I expect this should be a fairly
straightforward bugfix upgrade. Reviewers please to read the changelog
to see if there is anything we should be concerned about.

Change-Id: I4983865c94429c4cbcb54329f0f83b2fb0f26404
2024-09-23 08:44:08 -07:00
Zuul
74fc128f36 Merge "Run zuul-launcher" 2024-09-19 17:40:47 +00:00
James E. Blair
1c069add54 Run zuul-launcher
This doesn't do anything substantial yet, but we'd like to start running
the server soon in order to test it out.

Change-Id: I9eb2bccd6e5e9a064cbaff10676aeb1af6653f98
2024-09-18 16:37:40 -07:00
Clark Boylan
64d3f79fc4 Update etherpad to current develop branch state
This updates etherpad to the current develop branch state
(commit 08f199178d2932cc0ec956aaeb3f62e8a535598a) to pull in a fix after
the v2.2.4 release and before v2.2.5. Specifically we're interested in a
fix for embedding etherpad as we do with meetpad.

If a 2.2.5 release is pushed before this lands we should switch over to
that instead.

Change-Id: I497c6b434dae54ed808f62143a4c12fb42cc2c47
2024-09-13 08:56:29 -07:00
Clark Boylan
d03327853e Don't redirect etherpad manifest.json
After changes to etherpad in version 2.2.2 the way code is loaded from
the server has been changed. One of these changes appears to be the
loading of a manifest.json which currently gets redirected to
/p/manifest.json and is served like a pad... Serve this directly without
a redirect to /p/ so that we can return the correct content.

Change-Id: Ibd537ab371cf707e5121e48b0ab51e52046fed29
2024-09-11 09:40:21 -07:00
Clark Boylan
39d8d6ffb5 Update etherpad to 2.2.4
There are 2.2.0 and 2.2.1 tags but no built releases and they don't show
up in the changelog for 2.2.2 either. Thats fine we can ignore them and
upgrade to latest (2.2.4) instead. The changelog for 2.2.4 can be found
here:

  https://github.com/ether/etherpad-lite/blob/v2.2.4/CHANGELOG.md

Notable this changes how plugins are loaded into the js shipped to the
browser. We should confirm that our plugins are working as expected as
part of this update.

On the config management side of things there are some small updates to
the Dockerfile to sync up with upstream changes to how etherpad is
built. We also update the settings json file to configure log type. Note
this change was only made to the normal settings file and not the docker
settings file upstream so we match that in this change as well.

Finally we also update our mod_rewrite rules in apache to prevent new
javascript loading locations from being redirected to /p/
inappropriately. Previously we were redirecting foo.min.js to
/p/foo.min.js which caused the server to return html instead of js which
led to syntax errors. This then resulted in js errors from the
ep_headings plugin. It appears this plugin is ancient and no longer
maintained and seems to rely on require() functionality that was removed
from etherpad in 2.2.2. We switch to the ep_headings2 plugin instead.
This will allow us to file bugs against maintained software should
problems persist.

Fungi tested ep_headings2 against our production db content and things
seem to work despite this issue existing [0]. We should upgrade
carefully but it seems like things will likely be functional.

We should also check if these redirect rules affect meetpad as well. But
this can likely be done after the upgrade.

[0] https://github.com/ether/ep_headings2/issues/4

Change-Id: I4a907b5170d3612f4525153a0a07c291d6481a92
2024-09-09 08:45:36 -07:00
Clark Boylan
8cde4966a7 Tag etherpad images with version
We'll continue to deploy 'latest' but we tag the etherpad version
explicitly in order to make rollbacks simpler if necessary. Etherpad has
seen a resurgence in development which has led to some potentially
painful upgrade paths that we need to accomodate. Having rollbacks be
possible is a nice safety net.

Change-Id: I3ea59c1e4b33d777fae356d377773a4a60e9313e
2024-09-09 08:32:41 -07:00
Zuul
822f7d6ec5 Merge "Increase opendevzuul-subnet1 from /24 to /20" 2024-09-05 14:22:08 +00:00
Jeremy Stanley
9b9c835e09 Increase opendevzuul-subnet1 from /24 to /20
Since these are not globally routable addresses anyway, we might as
well use a larger network so that we're not constrained by the
subnet size later if we get more than 253 servers worth of quota.

This should be safe since the network is not presently in use for
anything outside of Rackspace Flex, which we haven't hooked up to
Nodepool yet. A cursory search of subset CIDRs also indicates
addresses in this range aren't being configured on virtual
interfaces for existing tests.

Change-Id: Ic32cbc0b24d037c67a5c2f8dd2834013017b8c87
2024-09-03 13:04:32 +00:00
Jeremy Stanley
cf33589c24 Add a Rackspace Flex SJC3 mirror server
Depends-On: https://review.opendev.org/927551
Change-Id: Ic193b95bd303c9b17c9920e979f583f736022aab
2024-08-30 00:34:00 +00:00
Jeremy Stanley
2d4103c521 Install dns-root-data before unbound
Since unbound refuses to start before dns-root-data is installed,
install it first.

Change-Id: I78f42c217b2627bf3108ac4b13d437bb03857222
2024-08-29 16:21:26 +00:00
Zuul
d1e6ca724c Merge "Set up networking for Rackspace Flex tenants" 2024-08-28 23:25:12 +00:00
Zuul
502d9549d8 Merge "Stop null-routing mail for edge-computing-owner" 2024-08-28 21:25:46 +00:00
Jeremy Stanley
da5a393cfe Set up networking for Rackspace Flex tenants
The default environment for Rackspace Flex requires user-created
Neutron networks. Add our custom subnets connected to the provider
network "PUBLICNET" with our usual keypairs and open security
groups.

This is based on Clark's change several years ago for the old
InMotion cloud: I2aed6dffde4a1d6e3044c4bd8df4ca60065ae1ea

Change-Id: I8878ff36381d1e82d3bb5180e72a7eec1ce28056
2024-08-26 22:22:15 +00:00
Zuul
5199013c30 Merge "Switch Rackspace Flex authentication to project_id" 2024-08-24 07:31:32 +00:00
Jeremy Stanley
546d6a0ed5 Switch Rackspace Flex authentication to project_id
For some reason (cache lookup timeouts?) using the project name and
domain wasn't working initially (but did begin working some time
after also logging into the Skyline dashboard). As a matter of
robustness, use the project IDs instead which worked immediately
with no problem.

Since we needed to add new values in our private hostvars for this,
go ahead and separate out the hostvars used for other items too as
future-proofing. These have all been added on the bridge now.

While we're here, do some cleanup of unnecessary default values
pointed out on the previous review.

Change-Id: I850ef61932e9818495fa99e1d13360693f82edd8
2024-08-23 17:30:03 +00:00
Jeremy Stanley
9ab9e7dc01 Add hostvars for new Vexxhost mirrors
Because we have different Let's Encrypt certs for each mirror
server, we need individual hostvars for those. Include them so that
the infra-prod-letsencrypt deploy works correctly again.

Also insert the associated Apache handlers for completeness.

Change-Id: I57712c1c528b9750c12efdd296cebf6fdb9a331f
2024-08-23 14:47:46 +00:00
Zuul
7caf61dc52 Merge "Set tox-linters job timeout to the default" 2024-08-23 12:41:58 +00:00
Zuul
12f96ff927 Merge "Add Rackspace Flex to our OpenStackSDK configs" 2024-08-23 12:41:56 +00:00
Zuul
849888407d Merge "Add inventory entries for new Vexxhost mirrors" 2024-08-23 12:41:54 +00:00
Jeremy Stanley
db1d4c8e23 Add Rackspace Flex to our OpenStackSDK configs
We're starting to experiment with Rackspace's new Flex cloud. It
uses basically the same credentials as their classic cloud but with
more typical Keystone configuration. Just add it to the clouds.yaml
configs initially so we can more easily interact with it from the
bridge, upload our server image and prepare to start launching a
mirror instance.

Because the credentials and identifiers are still basically the
same, this change relies on our existing private hostvars and
doesn't introduce any new ones for now.

Change-Id: I5d06a97d4ab44f02de59298a135c1f2384a2e18a
2024-08-22 21:54:28 +00:00
Clark Boylan
5059db691b Set tox-linters job timeout to the default
This job only takes a few minutes to run. There isn't any reason to use
an hour long timeout on this job as a result. Reset it back to the
default and simplify the config.

Change-Id: Id808e2fa35caa7cf627014b04d1560326eeebc5b
2024-08-22 13:33:05 -07:00
Zuul
757be14481 Merge "Run tox-linters on Noble" 2024-08-22 20:32:16 +00:00
Clark Boylan
2b33716d0f Run opendev-buildset-regitry on Noble
This unpins the nodeset for opendev-buildset-registry which should start
working once the parent change has landed and our new mirror content has
been copied and deployed. We'll need to recheck this change when we
believe we've reached that state.

Change-Id: I7a92036cc63963d3066b4eb2bf451dd0ec8a887b
2024-08-21 16:43:16 -07:00
Clark Boylan
9f78af3776 Run tox-linters on Noble
This uses a modern Hacking which is necessary to get a newer flake8
which can run on python3.12. We remove the nodeset pin at the same time
to ensure we get good test results. These tests will almost certainly
fail with all of the new rules are violating. Subsequent patchsets will
correct those.

Change-Id: Ifda62f61bdac870e7d9c0baa2f6a930d770ed101
2024-08-21 16:41:37 -07:00
Clark Boylan
279cb28e34 Add Noble packages to Docker mirror
Each Ubuntu and Debian release get a separate independent repository
(though served from the same AFS volume) which means we need to
explicitly add each release to reprepro when they occur. Now that we've
switched to Noble as our default nodeset it has become apparent we did
not do this for Noble's Docker packages.

Go ahead and add a debian docker noble mirror. Note that reprepro has a
*.log logrotate rule for all logs produced by reprepro so we don't need
to explicitly add a rule for the new log file. We also don't need a new
volume in AFS as a single deb-docker volume is used for all of these
repo mirrors.

We pin the tox-linters and buildset-registry jobs to jammy because they
don't currently run on Noble. Linters faile because we need to use newer
hacking whcih we'll do in a followup that cleans up new errors and the
registry fails due to the issue we are trying to fix in this very change
(need docker packages for noble).

Change-Id: I289f19a11539b490c3b7327d01d517948b95e072
2024-08-21 16:39:56 -07:00
Tony Breeds
74fe5e22fb Add inventory entries for new Vexxhost mirrors
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/925437
Change-Id: I41b306201dd45479a09e7584a720c66b3bc6f515
2024-08-20 19:55:00 -05:00
Zuul
394251ebdb Merge "Track our OpenMetal environment HTTPS cert expiry" 2024-08-20 20:05:54 +00:00
Zuul
469a15fb6f Merge "Add Noble nodes to system-config-run testing" 2024-08-20 19:24:50 +00:00
Jeremy Stanley
0f9a53d72c Track our OpenMetal environment HTTPS cert expiry
We don't manage the openmetal.us-east.opendev.org HTTPS certificate
with our usual Ansible/DNS/acme.sh process, so explicitly add it to
the certcheck list.

Change-Id: Icbcedb546036e7dadded352a9bdda91f57aa157f
2024-08-16 18:32:18 +00:00
Clark Boylan
1804f14ecb Force a rebuild of Gerrit images
We just updated the gerrit image version tags but that did not promote
the images in docker hub so production doesn't see them. Force a rebuild
via the dockerfile (with an updated comment) to actually get images to
promote.

Change-Id: I0ea50b1d92b8633e59d4c4aff1b0ec8c7a47a0b5
2024-08-15 11:26:44 -07:00
Clark Boylan
715cfa709d Update Gerrit image to 3.9.6
Update our Gerrit 3.9 image to 3.9.6 and our 3.10 image to 3.10.1. The
3.9.6 update is the one that will be deployed to production (3.10 is
only used for testing currently) and has the most relevant changes as a
result. The release notes for that release can be found here:

  https://www.gerritcodereview.com/3.9.html

There are bugfixes and performance updates. Notably the replication
plugins sees some updates. I don't expect these updates to affect our
use of the replication plugin but we should be on the lookout for any
unexpected behavior changes with this plugin.

Change-Id: Ida223104076bb161443269335a43efeb1bdf40e3
2024-08-09 08:59:15 -07:00
Jeremy Stanley
7640244fc0 Stop null-routing mail for edge-computing-owner
Years ago, while combating a rather nasty and prolonged bout of spam
to mailing list owner addresses, we added configuration to silently
drop any messages for them. That had a side-effect of also
discarding list moderation notifications. As the spam wave subsided
some time back and the primary manager of the edge-computing mailing
list would like to start receiving these notifications once more,
we're removing the line responsible from our listserv's MTA
configuration.

We could consider doing the same for other lists, but since the
sudden arrival of new notifications after years of silence may be a
surprise, we need to think about that more carefully before doing
so.

Change-Id: I10e371e22fd560f133445ce8d17f1c3a2698e839
2024-08-06 17:25:10 +00:00
Clark Boylan
e66eeb8c3c Remove most linaro cloud resources
This removes ansible configuration for the linaro cloud itself and the
linaro cloud mirror. This cloud is in the process of going away and
having these nodes in our inventory is creating base jobs failures due
to unreachable nodes. This then dominoes into not running the LE refresh
job and now some certs are not getting renewed. Clean this all up so
that the rest of our systems are happy.

Note that we don't fully clean up the idea of an unmanaged group as
there may be other locations we want to do something similar (OpenMetal
perhaps?). We also don't remove the openstack clouds.yaml entries for
the linaro cloud yet. It isn't entirely clear when things will go
offline, but it may be as late as August 10 so we keep those credentials
around as they may be useful until then.

Change-Id: Idd6b455de8da2aa9901bf989b1d131f1f4533420
2024-08-02 09:21:11 -07:00
Tony Breeds
c963023532 Add Noble nodes to system-config-run testing
Change-Id: I0c2fc6d19be1582479f3d4cf600dfe4d16920334
2024-08-02 08:48:23 +10:00
Tony Breeds
62b6ae4164 [base/unbound] Install dns-root-data package
Setting up a new Noble base server errors with:
  unbound[9702]: [9702:0] error: unable to open /var/lib/unbound/root.key for reading: No such file or directory
  unbound[9702]: [9702:0] error: error reading auto-trust-anchor-file: /var/lib/unbound/root.key

Rather that install and setup the root.key as described in[1] we install
the dns-root-data package which is recomended by the unbound package anyway

[1] https://nlnetlabs.nl/documentation/unbound/howto-anchor/

Change-Id: I6e6adffa8910931efa1f52d37848cce54f3b00c8
2024-08-01 10:02:18 +10:00
Zuul
f75e1443e5 Merge "Add vmware migration list to lists.openinfra.dev" 2024-07-22 16:57:17 +00:00