18009 Commits

Author SHA1 Message Date
Ian Wienand
4f3be693e5 gerrit docs: add note that duplicate user may have email addresses to remove
Change-Id: I54e935704d38dd60cdf75415dd8c61b06cae7b11
2022-06-24 15:37:54 +10:00
Ian Wienand
3a09bf7e8a gerrit docs: cleanup and use shell-session
A few formatting fixes

* try to more consistently use shell-session formatting for shell
  sessions (makes it easier to copy-paste).
* fix up and use more `` around verbatim/code things.

Fixes:

 * Gerrit Configuration : there's no db to set the ICLA fields in now,
   remove
 * Duplicate Accounts : add required arg "origin" to git fetch command
 * Deactivating account : can not delete comments via sql query,
   remove

Change-Id: Ia481750aa59fc88bef5c00bb0fd9e6f9e23b2777
2022-06-24 15:37:52 +10:00
Clark Boylan
761dbb4352 Fix zuul merger graceful stops
Previously the merger docker-compose restart value was set to always.
This caused the merger to immediately restart after asking it to
gracefully stop and our check for the merger stopping:

  docker-compose ps -q | xargs docker wait

never saw it as being stopped.

Make the mergers match executors and restart only on failure. This
should allow us to gracefully stop the mergers with intention and detect
they are stopped for maintenance purposes.

Change-Id: Ia8d12fbf6a45e4ca85174ccafd18b5d2351c26c1
2022-06-01 09:49:40 -07:00
Zuul
0e072f1399 Merge "Update Gerrit images to 3.4.5 and 3.5.2" 2022-06-01 02:04:47 +00:00
Clark Boylan
5cc6c14a6d Remove ethercalc config management
About a month ago we announced [0][1] that this server would be shutdown
and removed on May 31, 2022. Before we can shutdown the server we need
to remove it from config management to prevent Ansible errors. This
change is safe to land now, then on the 31st we can shutdown, snapshot,
and delete the server.

[0] https://lists.opendev.org/pipermail/service-announce/2022-May/000038.html
[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-May/028408.html

Change-Id: Ic44bed01384845e5b6322eeed02dd0932501cdb3
2022-05-30 12:57:48 -07:00
Clark Boylan
4cbc5ee254 Perform package upgrades prior to zuul cluster node reboots
This serves two purposes. The first is that not all packages are updated
by unattended-upgrades beacuse it may not be safe to upgrade packages
while services are running. We should be safe in this situation because
we've gracefully stopped services and can proceed with package updates.
The other is unattended upgrades runs daily which means we could end up
almost 24 hours out of date prior to rebooting. This ensures we have the
latest and greatest packages installed just prior to rebooting.

Change-Id: Id351b5478e925ed1b4fbb6b3e27f2c0b6af8b897
2022-05-26 14:04:24 -07:00
Zuul
383b1334b2 Merge "Add playbook to gracefully stop and reboot the zuul cluster" 2022-05-25 19:01:18 +00:00
Zuul
3c5046f718 Merge "Add the start of a Zuul rolling restart playbook" 2022-05-25 18:42:36 +00:00
Clark Boylan
88425158a1 Add playbook to gracefully stop and reboot the zuul cluster
This should simplify the process of applying patches to the cluster.

Change-Id: I28756e32c2f42186e11d78e4ca461e808026f632
2022-05-25 11:05:22 -07:00
James E. Blair
03a0bef7b6 Add the start of a Zuul rolling restart playbook
This handles rolling the mergers and executors, but not yet
the schedulers.

Also, it does the executors in complete batches of 6, but could be
improved to stop 6 and then do each of the next as the first ones
complete.

Change-Id: I2dca104194c2f129b68dcef7721d7d08cb987c46
2022-05-25 09:48:28 -07:00
Clark Boylan
819d3ce480 Update Gerrit images to 3.4.5 and 3.5.2
3.4.5 is a fairly minor update. Some bugs are fixed and jgit is updated.

3.4.5 release notes:
  https://www.gerritcodereview.com/3.4.html#345

3.5.2 is a bigger update and important adds support for being able to
upgrade to 3.6.0 later. There is a new copy-approvals command that must
be run offline on 3.5.2 before upgrading to 3.6.0. This copies approvals
in the notedb in a way that 3.6.0 can handle them apparently. The
release notes indicate this may take some time to run. We don't need to
run it now though and instead need to make note of it when we prepare
for the 3.6.0 upgrade.

3.5.2 release notes:
  https://www.gerritcodereview.com/3.5.html#352

For now don't overthink things and instead just get up to date with our
images.

Change-Id: I837c2cbb09e9a4ff934973f6fc115142d459ae0f
2022-05-25 08:37:33 -07:00
Zuul
977567ecef Merge "Add testing for jammy openafs" 2022-05-18 00:07:13 +00:00
Clark Boylan
5c50ae731f Upgrade gitea to v1.16.8
This appears to be a straightforward bug fix release according to the
release notes:

  https://github.com/go-gitea/gitea/blob/v1.16.8/CHANGELOG.md

No template change between v1.16.7 and v1.16.8 according to git.

Change-Id: I0b9bb2f15beb7d3b1541c02e6e96601d25449e33
2022-05-16 11:36:17 -07:00
Zuul
b4e1c3b7cb Merge "Stop mirroring source packages for debian" 2022-05-13 17:04:28 +00:00
Zuul
ca9b0f10f6 Merge "gitea-set-org-logos: openstack logo centered" 2022-05-13 01:26:43 +00:00
Ian Wienand
24e179d5de Add testing for jammy openafs
Change-Id: I733d10c9285d4ea0d86e97f6ed45f28376a8672b
2022-05-12 12:53:49 +10:00
Zuul
85465797f1 Merge "Remove group vars for removed services" 2022-05-11 23:27:04 +00:00
Zuul
e7f26a15f0 Merge "mirror-update: clean up Fedora ignores" 2022-05-11 05:29:45 +00:00
Ian Wienand
bcb73dd2ac mirror-update: switch Fedora mirror
In our onging quest to find a stable upstream source, switch this to
the facebook mirror which seems to have been working well enough for
centos.

While we're here, reduce the frequency a bit as there's no need to
sync every two hours; it's a bit unfriendly to the remote end.

Change-Id: I50f38b58b3f0c0557fb1e2e0667a17a4f1d1dec6
2022-05-11 09:57:46 +10:00
Clark Boylan
be3f421724 Fix opensuse upstream mirroring path
It appears our opensuse mirror upstream has added an extra opensuse/ in
the path to the opensuse packages that we mirror.

Currently we fail with:

  rsync: [sender] change_dir "/distribution/leap/15.2/repo" (in opensuse) failed: No such file or directory (2)

Browsing the repo via https we see the content exists at:

  https://mirror.clarkson.edu/opensuse/opensuse/distribution/leap/15.2/repo/

which includes an extra opensuse/ path segment. Now it is possible that
https and rsync have different root dirs, but considering the current
error trying this seems reasonable.

Change-Id: I37aa701af409a58d228c22d664ad0f49a23a1fc5
2022-05-10 14:00:53 -07:00
Zuul
813cc9b2ab Merge "Use add instead of wget to download things for gerrit images" 2022-05-10 06:14:26 +00:00
Zuul
869c49c1e6 Merge "Pull keycloak from quay.io" 2022-05-09 17:25:57 +00:00
Clark Boylan
6cd52a9ee4 Update etherpad to 1.8.18
This appears to be a minor bugfix update, but there is no reason for us
to fall behind. Changelog can be found at:

  https://github.com/ether/etherpad-lite/blob/1.8.18/CHANGELOG.md#1818

Change-Id: If675da1f3599898bddcc41a4aeb9ef9e9c2fc281
2022-05-06 09:09:36 -07:00
Clark Boylan
72eeb7548c Borg ignore ansible tmp files
If borg sees an ansible tmp file just before it is removed by ansible
then we can get errors of this form when running borg:

  stat: [Errno 2] No such file or directory: '/root/.ansible/tmp/ansible-tmp-$IDENTIFIER'

This causes the backup run to report failure. Address this by adding
/root/.ansible/tmp/* to our borg backup excludes list.

Change-Id: Ie2c7081a4510959f6514b1cb7eb8facc4ac129fb
2022-05-05 08:36:20 -07:00
Clark Boylan
93aaf92853 Pull keycloak from quay.io
Apparently upstream stopped updating docker hub images semi recently. We
should pull from the quay.io location instead. As far as I can tell
docker.io/jboss/keycloak and quay.io/keycloak/keycloak orginate from
https://github.com/keycloak/keycloak-containers so this switch is in
theory fine. But I guess we'll find out.

Change-Id: I42c4a62468a5960332126fec90fcf6307c5b6674
2022-05-04 13:08:28 -07:00
Zuul
14704cc8e0 Merge "Remove open-vm-tools from servers" 2022-05-04 06:40:45 +00:00
Zuul
538a9a3e84 Merge "etherpad: remove session key" 2022-05-04 06:27:45 +00:00
Zuul
86d58f6ce9 Merge "Test openafs roles on CentOS 9-stream" 2022-05-03 11:59:49 +00:00
Zuul
94f0241eff Merge "third-party CI: reminder to keep address current" 2022-05-03 00:01:30 +00:00
Zuul
3e7d049879 Merge "Clean up defunct OpenStack mailing lists" 2022-05-03 00:01:27 +00:00
Zuul
412f0fce3e Merge "Upgrade gitea to v1.16.7" 2022-05-02 23:24:44 +00:00
Jeremy Stanley
40caa4eec6 third-party CI: reminder to keep address current
Now that we're retiring the third-party-ci-announce mailing list,
which we never really used consistently anyway, just tell
third-party CI operators to make sure the E-mail address on their
account is current and reachable.

Change-Id: I6186149de25b06f2982702143a807de8bb01be73
2022-05-02 23:14:12 +00:00
Ian Wienand
b42769b7ed Test openafs roles on CentOS 9-stream
We have labeled the 8/9 stream repos with -stream for clarity; add
this to the path for the repo.

Change-Id: I5c4c5365d763f8a3c03a4adef36235e7809c44d7
Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/839689
2022-05-03 09:09:44 +10:00
Zuul
c61e83cd5d Merge "Explicitly disable Gerrit tracing.performanceLogging" 2022-05-02 18:39:14 +00:00
Zuul
c94e68597a Merge "Enable Gerrit httpd requestLog" 2022-05-02 18:37:44 +00:00
Zuul
69546bc70c Merge "Update Gerrit build checkouts" 2022-05-02 18:37:41 +00:00
Clark Boylan
0ae3ffc118 Upgrade gitea to v1.16.7
This is the latest bug fix release. Upgrade to it to pull in those bug
fixes. You can see the full list of changes in the release notes:

  https://github.com/go-gitea/gitea/blob/v1.16.7/CHANGELOG.md#1167---2022-05-02

I've checked template diffs between v1.16.6 and v1.16.7 and they are
empty.

Change-Id: I5c8cb5bf02ab5ca8f64da25f1384291921ba8bba
2022-05-02 10:10:11 -07:00
Zuul
03e9d43c97 Merge "Add Jammy Docker package mirroring" 2022-05-02 15:56:31 +00:00
Zuul
7e3e766f24 Merge "Mirror Jammy arm64 ubuntu-ports" 2022-05-02 15:56:18 +00:00
Clark Boylan
275fcba2e5 Explicitly disable Gerrit tracing.performanceLogging
According to the docs [0] this shouldn't be necessary as performance
logging only happens if a performance tracing plugin is installed.
However according to this repo discuss thread [1] there is always a
dummy performanceLogging instance installed. This same thread identifies
this as a likely source for large increase in memory utilization by
Gerrit when upgrading to 3.5.

Let's explicitly disable this tracing due to the memory overhead in prep
for our 3.5 upgrade. We can always flip the setting if we install a
performance tracing plugin in our Gerrit.

[0] https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#tracing
[1] https://groups.google.com/g/repo-discuss/c/QUD7_LsEVks/m/kBDEeam4AgAJ

Change-Id: Iff438695aa6488fb5886120121946494b1edf003
2022-04-29 12:02:17 -07:00
Clark Boylan
c7874b5770 Enable Gerrit httpd requestLog
Because we proxy to Gerrit and set listenUrl with a proxy-http:// prefix
httpd.requestLog is disabled by default. We choose to explicitly enable
it here to add more logging to the Gerrit system even if this logging is
slightly less useful when behind a proxy. In particular this logging
will track memory utilization per request which we can use to benchmark
change query memory cost between 3.4 and 3.5.

Change-Id: Ia3ccf820ee0e5ca7d68bcc37da7004dea2ad7128
2022-04-29 12:01:52 -07:00
Jeremy Stanley
1f1443a2c8 Clean up defunct OpenStack mailing lists
In preparation for retiring a number of mailing lists from
lists.openstack.org which have had no activity for over three years,
remove their configuration so our deployment automation won't
recreate them once they're gone. Also remove references to the
third-part-announce list in our documentation, since that's one of
the unused lists we're removing. See the announcement at
http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027404.html
for details.

Change-Id: Ieedd8613363039d19d3ae47f1a83a38747419bdc
2022-04-29 18:27:22 +00:00
Clark Boylan
a285969ffc Mirror Jammy arm64 ubuntu-ports
This will enable us to spin up Ubuntu Jammy arm64 images that run jobs
using our mirrors.

Change-Id: Ie34065f93191b52ee5cf20d380a266754c7d4cfe
2022-04-29 09:34:59 -07:00
Jeremy Stanley
d185aedd7d Decommission status.openstack.org and services
The status.openstack.org server is offline now that it no longer
hosts any working services. Remove all configuration for it in
preparation for retiring related Git repositories.

Also roll some related cleanup into this for the already retired
puppet-kibana module.

Change-Id: I3cfcc129983e3641dfbe55d5ecc208c554e97de4
2022-04-29 16:34:51 +00:00
Clark Boylan
e95fcef6a6 Remove health group from our ansible groups
The openstack health server was removed but cleaning up its groups was
missed. Fix this by removing the group from our ansible configs.

Change-Id: I84fdf290e359609fc0854719fa9ecb032c7ea254
2022-04-29 16:33:33 +00:00
Ian Wienand
9f4fa24025 Remove puppet-kibana
I think this was overlooked in the removal of the ELK stack with
I5f7f73affe7b97c74680d182e68eb4bfebbe23e1, the repo is now retired.

Change-Id: I87bfe7be61f20a7c05c500af4e82b787d9c37a8c
2022-04-29 17:35:44 +10:00
Zuul
c0e8221331 Merge "Stop mirroring source packages for ubuntu" 2022-04-28 22:55:20 +00:00
Zuul
a189bb96aa Merge "Stop mirroring source packages for debian-docker" 2022-04-28 21:03:26 +00:00
Jeremy Stanley
768dfc924c Stop mirroring source packages for debian
As an overall clean-up effort to free space on our mirror volumes,
stop mirroring source packages for the debian package repository.
According to codesearch, the configure-mirrors role in
zuul/zuul-jobs adds deb-src lines for it, so we need to wait on
merging this until a change merges to switch its default behavior or
we override the default behavior in opendev/base-jobs. It also won't
take effect immediately, and will require a manual step to prune the
old packages from the filesystem once this configuration update has
deployed.

Change-Id: Ia5777770e178385b7326c97728c875f302c1e1f0
Depends-On: https://review.opendev.org/839593
2022-04-27 19:30:17 +00:00
Jeremy Stanley
1474d10652 Stop mirroring source packages for ubuntu
As an overall clean-up effort to free space on our mirror volumes,
stop mirroring source packages for the ubuntu package repository.
According to codesearch, no jobs in our system add deb-src lines for
it so this should be entirely safe. It also won't take effect
immediately, and will require a manual step to prune the old
packages from the filesystem once this configuration update has
deployed.

Change-Id: Ie2aa6bdf43721c9ceacea54f7f5bc928883287a7
2022-04-27 19:29:50 +00:00