19162 Commits

Author SHA1 Message Date
James E. Blair
1c069add54 Run zuul-launcher
This doesn't do anything substantial yet, but we'd like to start running
the server soon in order to test it out.

Change-Id: I9eb2bccd6e5e9a064cbaff10676aeb1af6653f98
2024-09-18 16:37:40 -07:00
Zuul
f75e1443e5 Merge "Add vmware migration list to lists.openinfra.dev" 2024-07-22 16:57:17 +00:00
Clark Boylan
8f90bd37f1 Update test gitea's JWT_SECRET
Related to the parent change we want to ensure that the test deployment
of gitea is using a valid secret. This new value was generated via:

  `sudo docker exec gitea-docker_gitea-web_1 gitea generate secret JWT_SECRET`

On a held test node and should be more valid than the previous value.

Change-Id: I8cbbaf061e168c2874eb244ff18da6ee544e0d80
2024-07-19 11:27:10 -07:00
Clark Boylan
4a3c19e053 Fix the formatting of the gitea app.ini file
During the Gitea 1.22.1 upgrade we had failures because gitea wanted to
write back to its app.ini config file and couldn't do so due to file
permissions. We address that by fixing the formatting for one entry in
the template then separately update a secret var to match what ended up
being deployed to gitea09. In theory this should make everything line up
such that we don't actually need to write back to the file at all.

If this continues to fail we will instead update file permissions and
let gitea rewrite its file instead. But that would be the next thing to
try.

Change-Id: Ic3a6e2dd25ec906c234c2fea26b68eb8f742201c
2024-07-19 10:23:54 -07:00
Zuul
4b31c5409d Merge "Update Gitea to version 1.22" 2024-07-19 16:54:37 +00:00
Clark Boylan
0d23743dc9 Add vmware migration list to lists.openinfra.dev
This has been requested by Jimmy at the foundation.

Change-Id: I997783b29b98dc001152d1fd8e3d8f439caece9a
2024-07-18 12:03:33 -07:00
Clark Boylan
847a835d3a Limit linaro mirror Apache cache size
The linaro mirror node only has a 100GB cache volume to be shared
between OpenAFS and Apache. Each cache gets a ~50GB filesystem dedicated
to it. Unfortunately we were telling Apache's cache pruning system to
limit to 60000M which is large than 50GB. THis resulted in the cache
filling up and I think it may also be the source of errors.

Address this my reconfiguring htcacheclean to limit to ~40GB. This limit
is set with some buffer space as htcacheclean runs periodically and has
to keep things low enough to avoid filling up.

Change-Id: I7af3daa396b1631feefed5952460b8a9710b25b5
2024-07-12 11:29:36 -07:00
Zuul
a6ef5c87aa Merge "Fix task name to reflect correct distro" 2024-07-11 14:06:50 +00:00
Clark Boylan
352067b5af Pin the Jaeger container to 1.58.1
Jaeger just released 1.59.0 and that appears to have broken Zuul's
quickstart again. Pin to the 1.58.1 release which should be what we were
running before the 1.59.0 update out of an abundance of caution.

Change-Id: Iefed34f8df2eb105efa1b68edb5d5c76104f836f
2024-07-10 11:43:55 -07:00
Tony Breeds
500fc92f17 Fix task name to reflect correct distro
Change-Id: I4ac5bdd6ab27c460b8864b75157a8487128d5a76
2024-07-09 09:12:13 +10:00
Clark Boylan
2192d2d62d Upgrade to etherpad 2.1.1
This newer version actually restores APIKEY authentication, but we
already converted to oauth2.0 so we don't revert. Otherwise it seems
like there are a number of small fixes. Full change log here:

  https://github.com/ether/etherpad-lite/blob/v2.1.1/CHANGELOG.md

In this change we resync configuration template files which results in a
few small updates. We also realign the dockerfile with upstream which
also results in a few small updates one of which is bumping the nodejs
version to 22 from 20.

Change-Id: I39664fde59a7cc9fdf2451d41018ae11b9e99b79
2024-07-08 09:02:54 -07:00
Clark Boylan
a5095d52bc Update Gitea to version 1.22
Changes made on our side to make this upgrade happen:

* Update the gitea checkout tag to v1.22.1
* Update the golang container version to 1.22 as gitea 1.22 has an
  undocumented hard dependency on golang 1.22 or newer.
* Update our overridden template files to match latest gitea template
  changes.
* Update our app.ini config to switch from [oauth2].ENABLE to
  [oauth2].ENABLED as the previous config string is deprecated and will
  be removed in 1.23.0 per:
    ...es/setting/oauth2.go:124:loadOAuth2From() [E] Deprecation: config
    option `[oauth2].ENABLE` presents, please use `[oauth2].ENABLED`
    instead because this fallback will be/has been removed in v1.23.0

The full release notes for this release can be found here:

  https://github.com/go-gitea/gitea/blob/v1.22.1/CHANGELOG.md

I've including the list of breaking changes below with my own
annotations on how/whether they affect us.

* BREAKING
  * Improve reverse proxy documents and clarify the AppURL guessing behavior (https://github.com/go-gitea/gitea/pull/31003) (https://github.com/go-gitea/gitea/pull/31020)
    * This isn't actually a breaking chagne but they have improved docs
      around how to properly set Host and X-Forwarded-Proto headers for
      gitea to enable better logging behind a reverse proxy. We should
      investigate.
  * Remember log in for a month by default (https://github.com/go-gitea/gitea/pull/30150)
    * Default was a week. We should consider rolling back to low values
      since we don't have real users.
  * Breaking summary for template refactoring (https://github.com/go-gitea/gitea/pull/29395)
    * All custom templates need to follow these changes
    * I don't think we're using any of the changed methods/functions in
      our templates. Testing should help confirm this.
  * Recommend/convert to use case-sensitive collation for MySQL/MSSQL (https://github.com/go-gitea/gitea/pull/28662)
    * This is the doctor update to address case sensitivity problems
      between git and gitea. We'll need to test this as part of our
      upgrade process and testing.
  * Make offline mode as default to not connect external avatar service by default (https://github.com/go-gitea/gitea/pull/28548)
    * We are already disabling gravatar. I think this will disable it
      harder.
  * Include public repos in the doer's dashboard for issue search (https://github.com/go-gitea/gitea/pull/28304)
    * This affects end user dashboard info rendering which we don't use.
  * Use restricted sanitizer for repository description (https://github.com/go-gitea/gitea/pull/28141)
    * We already control what goes into repo descriptions via
      projects.yaml. Shouldn't really affect us.
  * Support storage base path as prefix (https://github.com/go-gitea/gitea/pull/27827)
    * This change looks scary at first glance but appears to only affect
      minio storage systems (which is like an s3 abstraction layer). We
      store things to disk and shouldn't be affected if I read the PR
      correctly.
  * Enhanced auth token / remember me (https://github.com/go-gitea/gitea/pull/27606)
    * THis appears to improve security but it isn't clear what the
      effect on end users is. We'll see if our CI jobs are happy with
      new token generation I guess.
  * Rename the default themes to gitea-light, gitea-dark, gitea-auto (https://github.com/go-gitea/gitea/pull/27419)
    * If you didn't see the new themes, please remove the [ui].THEMES config option from app.ini
    * We don't do anything special for themes so this should noop for
      us.
  * Require MySQL 8.0, PostgreSQL 12, MSSQL 2012 (https://github.com/go-gitea/gitea/pull/27337)
    * Our version of MariaDB should be new enough to rough rough feature
      equivalent with MySQL 8.0 and newer. We might consider helping
      upstream add MariaDB testing if they haven't already though.

Change-Id: Ifb4f0d92d70bc06f717e6535f1b67a221e127180
2024-07-07 19:30:42 -07:00
James E. Blair
84bda251dd Use jaeger all-in-one v1 image
It looks like the :latest image may be the head of the main branch,
which is cool, we do that too, but apparently Jaeger does not test
their releases with Zuul, and it appears they have merged something
that breaks our usage for it.  They may be starting v2 development
and thus may be intentionally merging backwards incompatible changes.

To address this, use the :1 image tag, so that we are still compatible
with Zuul.

Change-Id: I7f9602fbd2d98502c5c88641f61b4e408aa1525d
2024-07-03 11:33:46 -07:00
Tony Breeds
19049bcde5 Test mirror services on noble
Change-Id: I3f9fa9aae086d993c826dbabd707be449b70ea45
2024-06-28 08:15:41 +10:00
Tony Breeds
2433597e81 Don't use the openstack-ci-core/openafs PPA on noble and later
Also add zuul-role-integration jobs for noble to ensure things generally
work as expected on noble, and specifically verify from the testing logs
that the exclusion is effective.

Change-Id: Ic722e992038b3d740108594d80d66f0befc153c7
2024-06-28 08:11:15 +10:00
Zuul
487b29e013 Merge "Remove *zuul-role-integration-centos-8-stream* jobs" 2024-06-26 22:44:15 +00:00
Zuul
febc91d20e Merge "Add noble repo files" 2024-06-26 22:44:13 +00:00
Zuul
723a7a4d0d Merge "Remove centos rsync mirroring tooling" 2024-06-26 22:01:32 +00:00
Zuul
09934f3230 Merge "Delete centos 8-stream mirror content" 2024-06-26 20:10:01 +00:00
Zuul
11279f908b Merge "Drop CentOS 8 Stream jobs" 2024-06-26 19:39:26 +00:00
Jeremy Stanley
1f10a66956 Correct the Exim queue threshold for Mailman
In change I9e21eb19b85c9a4b19f5e6c5d03a89c89028bbc4 we incorrectly
adjusted Exim's smtp_accept_max_per_host option from 10 to 50 rather
than setting the smtp_accept_queue_per_connection option we actually
needed. Add this parameter as a new tunable in the role, set the
other option back to its prior value, and update comment references
accordingly.

Change-Id: I13f0275202eba8b5190a76bff921f1ac5adbeea0
2024-06-26 15:39:50 +00:00
Clark Boylan
195e76fa07 Remove centos rsync mirroring tooling
Now that CentOS 7, 8 and 8-stream have been removed from mirroring we
can drop the mirror script for these releases entirely. Note that
9-stream and beyond use the newer centos-stream script instead and are
not affected.

Once this is completed we should be able to clean up the afs volume for
centos entirely as well (though it is empty currently and has a very low
cost in that state).

Change-Id: I744c61e3386d4ea54efaae9c3a46fd08878263ff
2024-06-25 10:02:52 -07:00
Clark Boylan
dcec64c67b Delete centos 8-stream mirror content
There is a README and a few empty directories to clean up from the
8-stream mirror content. This should be safe to land at any time since
the mirror is no longer functional after upstream cleanup that we
synced.

Change-Id: I467c1f575e9461a8bf8d37697161e65114e5161a
2024-06-25 09:12:03 -07:00
Jeremy Stanley
cca985c25b Rebalance Mailman's and Exim's outgoing batch size
In order to avoid tripping Exim's threshold for punting deliveries
to periodic queue processing, make sure to set Mailman's
mta.max_recipients value less than Exim's smtp_accept_max_per_host.
This should eliminate the "no immediate delivery: more than X
messages received in one connection" errors in Exim's mainlog.

While we're at it, increase both for greater throughput.

Change-Id: I9e21eb19b85c9a4b19f5e6c5d03a89c89028bbc4
2024-06-25 03:16:51 +00:00
Clark Boylan
e501070ad9 Drop CentOS 8 Stream jobs
This distro release has gone eol and the package repos were
automatically cleaned out for us by upstream. This has resulted in jobs
no longer functioning. Cleaning out the jobs is the first step in
eventually removign the nodesets and image builds from zuul+nodepool.

Change-Id: I873c2b719bad1e9c7b482a42a8566b6b06d447fc
2024-06-24 08:56:13 -07:00
Clark Boylan
4a631dd201 Cleanup leftover inmotion configs
This removes leftover inmotion clouds.yaml related configs from our
configuration management. We remove the hardcoded cert (since the old
cloud used a self signed cert) and various ansible vars used in testing.
Note we keep the ansible task to write out special cloud certs so that
we don't have to reinvent this code in the future / remember we had it
once and unrevert it, but we feed that task an empty list making it a
noop currently.

Change-Id: I2a6fdf06879023ab6ee65437a592d55a8c0eb371
2024-06-20 09:52:50 -07:00
Zuul
fdfa10ded9 Merge "Remove old meetpad and jvb servers" 2024-06-20 02:31:41 +00:00
Tony Breeds
b33002dc42 Remove *zuul-role-integration-centos-8-stream* jobs
Change-Id: I9f178a972e7f1ed0a845f29e0e563e448f9d5000
2024-06-20 11:56:25 +10:00
Tony Breeds
2d32f57750 Add noble repo files
Change-Id: Ic5c1f35dc5417f0b46342d969c6b1d4841232758
2024-06-20 11:56:25 +10:00
Zuul
0edbecd516 Merge "Add golang based docker compose tool." 2024-06-19 13:08:46 +00:00
Zuul
c147e0fce1 Merge "Report Zuul events for WanderTracks repos" 2024-06-19 11:05:29 +00:00
Tony Breeds
25727fab47 Add golang based docker compose tool.
This installs the golang based compose (v2) tool.  Also make it the
python based tool optional, but installed by default.

The docker compose v2 tool comes from either:
 - Disrto   : Moby Engine : docker-compose-v2     : Focal -> Oracular
 - Upstream : Docker Inc  : docker-compose-plugin : Trusty -> Noble

With manual checking I can't see any distro older than focal that makes
use of the install-docker role, and all in-tree users pull from
upstream, so I think adding the golang compose tool without checking
the release is safe.

Change-Id: Ib8b5af40e626cd2d1a12b643f348001b9fdf4586
2024-06-19 11:26:26 +10:00
Tony Breeds
0ab443dddb Add openmetal mirror node to the inventory
Depends-On: https://review.opendev.org/c/opendev/zone-opendev.org/+/921895
Change-Id: I5323b088bdb6444c366f612415e72a3cebd5c988
2024-06-12 16:42:16 -05:00
Tony Breeds
2ed255798f Add OpenMetal to cloud-launcher
Change-Id: Ifcd62c5e41294da6a7912959483bea5a1cf08738
2024-06-11 17:45:05 -05:00
Tony Breeds
125f0b81fa Add configuration for new OpenMetal cloud
Change-Id: I1f84bb23a42202af48effb62330413cb2afaab65
2024-06-11 11:40:05 -05:00
Monty Taylor
2ed7e00706 Report Zuul events for WanderTracks repos
Now that we're properly Zuuling the things, it would be great to get
Matrix messages about it. Yay for distributed chat systems!

Change-Id: I2f355b98d93ae43aad72e80532450054caeec35b
2024-06-11 08:30:11 -07:00
Clark Boylan
faf32b452d Mark .git dirs safe when marking ansible roles safe for git
In the previous change we set the top level ansible role git repo dir as
safe for git to perform clones against. It seems that for this to work
the way we want it to we actually need to mark the .git dir as safe
instead. Go ahead and do so here.

Change-Id: Id877c721e94d015f68cb51fb862b220da0414970
2024-06-10 08:15:44 -07:00
Zuul
cfc5e72479 Merge "Mark source repos as safe in install-ansible-role" 2024-06-07 19:00:47 +00:00
Zuul
136d3be3c0 Merge "Add Gerrit 3.10 upgrade testing" 2024-06-07 16:14:54 +00:00
Zuul
f1093da7f8 Merge "Add Gerrit 3.10 image builds and testing" 2024-06-07 16:14:52 +00:00
Zuul
2ec5aa70d7 Merge "Remove Gerrit 3.8 images and related jobs" 2024-06-06 21:04:19 +00:00
Zuul
548b697ddb Merge "Fixups: for jitsi-meet" 2024-06-06 19:07:36 +00:00
Zuul
037c965528 Merge "Increase the number of mailman3 outgoing runners to 4" 2024-06-05 18:56:10 +00:00
Zuul
7d800c045b Merge "Remove inmotion cloud management" 2024-06-05 18:56:08 +00:00
Clark Boylan
a4c79cdd07 Increase the number of mailman3 outgoing runners to 4
By default mailman3 uses a single outgoing mail runner. We have seen
lists with large membership (openstack-discuss for example) delay mail
delivery for 5-10 minutes. In an attempt to increase throughput we bump
the number of outgoing runner instances to 4.

This upstream thread [0] has more info on this and other tuning options
we might consider. We start with this one as the other options have
behavior implications and will need more thought.

[0] https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/W4F2OEM4VIEEU4U6MAPVMFIEGERBMY55/

Change-Id: I9e353960fe55468be1dadffd37aada7da6ed7db0
2024-06-05 09:41:02 +00:00
Clark Boylan
f679683728 Add Gerrit 3.10 upgrade testing
This adds upgrade testing of gerrit from 3.9 to 3.10. This is the very
first step in the very long process of eventually upgrading to Gerrit
3.10.

Change-Id: Iaa77287b1334dc761e8de815704fec4d8d69d863
2024-06-04 13:30:19 -07:00
Clark Boylan
400514a1b6 Add Gerrit 3.10 image builds and testing
This adds Gerrit 3.10 image builds and our system-config-run-gerrit jobs
for Gerrit 3.10.

Change-Id: I6525090f78a1f1440f2cccda96f83158cb08d2ac
2024-06-04 13:29:25 -07:00
Tony Breeds
716eff81f1 Fixups: for jitsi-meet
When updating jitsi-meet to jammy we switched to using {meetpad,jvb}99
as the hostnames but were setting host-vars for {meetpad,jvb}01.  Fix that
Also a pointless and harmless ')' was left in a comment so clean that up.

Change-Id: Iad3079d9295559fd286bb41361f4de26efc14091
2024-06-04 13:43:15 -05:00
Clark Boylan
8c4977181b Remove inmotion cloud management
This removes management of the inmotion cloud mirror and cloud launcher
configs in prepration for retirement of this cloud. We don't remove the
cloud from clouds.yaml files as it is a bit more ambiguous as to how
long that will be useful (potentially necessary for manual cleanup
steps). Instead when we get around to adding openmetal after inmotion
has been shutdown and resurrected as a new openmetal cloud we can
replace clouds.yaml config then.

This cleanup is necessary to avoid errors when the cloud goes away. We
will be working with OpenMetal to make this happen. It shouldn't matter
if we land this before or after the project-config changes for nodepool
cleanup as things are decoupled sufficiently well.

Change-Id: I9d224318a9cfac35b867babff92e1071ca23c574
2024-06-03 13:21:31 -07:00
Clark Boylan
ed9aae01c3 Mark source repos as safe in install-ansible-role
Our install-ansible-role role is used to install the cloud launcher and
puppet ansbible roles on bridge. It does this by cloning the zuul repos
for these roles into the appropriate ansible role path location. Recent
versions of git will not clone repos owned by different users by default
due to security concerns. This breaks install-ansible-role because zuul
owns the zuul repos and install-ansible-role is cloning as root. In this
case we do trust the zuul repos though so we mark them as safe
directories in the root .gitconfig file.

Change-Id: I0cc4a011bbfb484fcc2ccf0d8b1d254c01fc03d6
2024-06-03 12:27:04 -07:00