331 Commits

Author SHA1 Message Date
Clark Boylan
e2442eeaf0 Don't run infra-prod-run-refstack on all group var updates
This was running on all group var updates but we only need to run it
when refstack group vars update. Change the file requirements to match
the refstack.yaml group file to address this.

Change-Id: Id5ed4b65c1ed6566696fea9a33db27e9318af1a6
2022-03-04 15:30:47 -08:00
James E. Blair
3f8acefbe1 Run zuul-web on zuul01 and add to load balancer
Change-Id: Ia8b10338fa3a1876993404276e0759f4b10d6b54
2022-03-04 13:11:09 -08:00
Clark Boylan
47c242ff21 Pull gerrit/plugins/gitiles from stable branch not tag
This plugin was updated to accomodate the ${hash} substition in gerrit
gitweb weblinks. We now need this updated version to build Gerrit
successfully but there is no tag for it yet. Just use the branch to
address this.

Change-Id: I4b0fd4ac845cc4289f78aacfa536db4185f12d38
2022-03-03 10:48:03 -08:00
Zuul
012ba26d38 Merge "encrypt-logs: turn on for all prod playbooks" 2022-03-01 19:37:05 +00:00
Ian Wienand
25f7403e2a hound: enable detect-ref
The dependent change enables the "detect-ref" option of hound, which
looks at the remote origin HEAD and indexes on that.  That should
allow indexing of our mixed repos that have a mix of "master" and
"main".

Add cirros to the test, which should exercise this path, and take some
screenshosts because this a js/react app and just a "curl" doesn't
help.

Change-Id: I1850577c63566b594f9730f5b8f0bc10b07ff7e4
Depends-On: https://review.opendev.org/c/opendev/jeepyb/+/830919
2022-02-25 17:27:35 +11:00
Ian Wienand
3f6cd427d7 encrypt-logs: turn on for all prod playbooks
We have validated that the log encryption/export path is working, so
turn it on for all prod jobs.

Change-Id: Ic04d5b6e716dffedc925cb799e3630027183d890
2022-02-24 09:57:55 +11:00
Zuul
65b20bc8d5 Merge "run-production-playbook: return encrypted logs" 2022-02-21 00:29:57 +00:00
Zuul
82c36fc784 Merge "Base work for exporting encrypted logs" 2022-02-21 00:29:54 +00:00
Ian Wienand
7b22badf6a run-production-playbook: return encrypted logs
Based on the changes in I5b9f9dd53eb896bb542652e8175c570877842584,
enable returning encrypted log artifacts for the codesearch production
job, as an initial test.

Change-Id: I9bd4ed0880596968000b1f153c31df849cd7fa8d
2022-02-16 16:39:46 +11:00
Ian Wienand
ccf00b7673 Base work for exporting encrypted logs
Our production jobs currently only put their logging locally on the
bastion host.  This means that to help maintain a production system,
you effectively need full access to the bastion host to debug any
misbehaviour.

We've long discussed publishing these Ansible runs as public logs, or
via a reporting system (ARA, etc.) but, despite our best efforts at
no_log and similar, we are not 100% sure that secret values may not
leak.

This is the infrastructure for an in-between solution, where we
publish the production run logs encrypted to specific GPG public keys.

Here we are capturing and encrypting the logs of the
system-config-run-* jobs, and providing a small download script to
automatically grab and unencrypt the log files.  Obviously this is
just to exercise the encryption/log-download path for these jobs, as
the logs are public.

Once this has landed, I will propose similar for the production jobs
(because these are post-pipeline this takes a bit more fiddling and
doens't run in CI).  The variables will be setup in such a way that if
someone wishes to help maintain a production system, they can add
their public-key and then add themselves to the particular
infra-prod-* job they wish to view the logs for.

It is planned that the extant operators will be in the default list;
however this is still useful over the status quo -- instead of having
to search through the log history on the bastion host when debugging a
failed run, they can simply view the logs from the failing build in
Zuul directly.

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/828818/
Change-Id: I5b9f9dd53eb896bb542652e8175c570877842584
2022-02-16 16:39:42 +11:00
Clark Boylan
1bee6ec357 Stop using puppet repos that will be retired
This should act as a sanity check that the puppet modules that we are
planning to retire are not used. The jobs updated here run puppet noop
applies and should confirm we don't have transitive needs for this
modules.

Change-Id: Ie4c7b809b22e9bded65a17876a9eb98195fc8910
2022-02-14 22:33:27 +00:00
James E. Blair
f196aa047e Clean up some gitea-lb zuul config
This triggers the test job on changes to any gitea.* roles, including
gitea-lb which wasn't included before.

It also removes the letescrypt job as a soft dependency from the lb
jobs since that is not strictly necessary.

Change-Id: Ie5bcd4d8215bb14d939dddf3e20d3173ccc0acdc
2022-02-10 23:38:59 +00:00
James E. Blair
2a9553ef25 Add Zuul load balancer
This adds a load balancer for zuul-web and fingergw.

Change-Id: Id5aa01151f64f3c85e1532ad66999ef9471c5896
2022-02-10 13:24:42 -08:00
Clark Boylan
a5671ddaf3 Update infra-prod-service-review dependencies
We removed the promote jobs for Gerrit 3.3 images but left them in place
as infra-prod-service-review dependencies. Fix that by updating the
infra prod job dependencies to the job for Gerrit 3.4 image promotion.

Change-Id: If2277799db91ea61aaffafb600f403531a0fb562
2022-02-07 13:03:50 -08:00
Clark Boylan
a2dce17612 Test Gerrit upgrade from 3.4 to 3.5
This reenables Gerrit upgrade testing but tests the 3.4 to 3.5 upgrade
now. Note this may need some work to get happy once we have 3.5 images
which is why we've split it out into a separate change.

Change-Id: Ibbbd3f98ac2df8d99d4ffda57df59f4a47da3cd3
2022-02-03 08:11:31 -08:00
Clark Boylan
a3cc983502 Add Gerrit 3.5 image builds and testing
This will build gerrit 3.5 images and run it through our standard Gerrit
testing. Upgrade testing from 3.4 to 3.5 to follow in followup changes.

Change-Id: I76d0389d1455e62b242aad1926b3a09830301801
2022-02-03 08:09:14 -08:00
Zuul
fd3cd75f7c Merge "Stop building Gerrit 3.3 images" 2022-02-03 05:50:59 +00:00
Zuul
72eac7bf9b Merge "Switch nb01 to focal in testing" 2022-02-03 00:36:10 +00:00
Zuul
f339b7cdab Merge "Better organize gerrit plugins in job defs" 2022-02-02 23:51:23 +00:00
Clark Boylan
ba562705c0 Switch nb01 to focal in testing
These nodes are focal in production now. Update testing to match.

Change-Id: Id0a0c784dd34ce9ac37df3715972bb9e3103bed8
2022-02-02 15:29:31 -08:00
Clark Boylan
4e9da3c255 Stop building Gerrit 3.3 images
We've upgraded to 3.4 and don't appear to be reverting. Remove the 3.3
images as they are no longer needed.

Note we comment out the review upgrade testing jobs until we have 3.5
images building.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/827562
Change-Id: I0e3cb81b790ab06c690ed0245526e4f47911c584
2022-02-02 14:03:28 -08:00
Ian Wienand
545feac217 infra-prod-grafana: drop system-config-promote-image-grafana
We dropped making our own grafana container with with
If0d584f848f213aeea385885e3decfaee6303de5, so we don't need this job
any more.

Change-Id: Ide212f25cda6d25e5cc31b0e8d2a65f3759bafdd
2022-01-31 17:13:40 +11:00
James E. Blair
1492f22faa Use grafyaml container image
Instead of building a local grafana image with grafyaml installed,
use the plain upstream grafana image along with the newly created
separate opendev grafyaml image to run the dashboards.

Depends-On: https://review.opendev.org/780119
Change-Id: If0d584f848f213aeea385885e3decfaee6303de5
2022-01-20 09:25:00 +11:00
Zuul
4fdb4907fd Merge "Update Gerrit images to 3.3.9 and 3.4.3" 2022-01-18 17:39:11 +00:00
Zuul
66e96dbd93 Merge "Remove centos-8 role integration testing" 2022-01-13 19:34:58 +00:00
Zuul
d2016bcba0 Merge "Update refstack image to bullseye" 2022-01-12 17:50:17 +00:00
Clark Boylan
0e10f3395e Remove centos-8 role integration testing
This testing is primarily for the openafs role. We may need
https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/824236 to
land first and update packages specifically for centos-8-stream before
this is mergeable. But CentOS 8 is EOL so we don't have much choice.

Change-Id: Idbd85c8f49cd169ee17f1b09db2de8bd36a52950
2022-01-11 13:08:48 -08:00
Clark Boylan
7f5f0bd4df Update refstack image to bullseye
Some spring cleaning now that bullseye is available. While we are at it
we fix up the job requires and provides for the related image building
and service testing jobs too.

Change-Id: I8a392e06df66b2e0c85157e61e529bb649d8ad58
2022-01-10 11:20:24 -08:00
Clark Boylan
de10234b9b Better organize gerrit plugins in job defs
The comments around the organization of these plugins wasn't accurate.
Instead of grouping them by git repo state group them by what
functionality they affect.

Change-Id: I6a21574e7079b7ddda520f727bef5562e5999126
2022-01-10 09:17:42 -08:00
Clark Boylan
3fdec9d20f Update Gerrit images to 3.3.9 and 3.4.3
This includes the fixes for right to left unicode that we've already
picked up, but now in an official release. It also updates Elasticsearch
support to prevent use with elasticsearch versions that are vulnerable
to log4shell. We don't use elasticsearch with Gerrit so this doesn't
affect us.

Change-Id: Ifbba7391bd16bc20c003293030a71e0ea787d0c7
2021-12-27 11:41:56 -08:00
Jeremy Stanley
81f8cdfb7b Add HTTPS vhosts to mailman servers
Add secondary vhosts for HTTPS to each mailman site, but don't
remove the plain HTTP ones for now. Before switching to Mailman 3
we'll replace the current HTTP vhosts with blanket redirects to
HTTPS.

Add tests to make sure this is working, and also add a command-line
test for the lists.openinfra.dev site now that it's got a first
non-default list of its own. Also collect Apache logs from the test
nodes so we can see for sure what might break.

Change-Id: I4d93d643381f17c9a968595587909f0ba3dd6f92
2021-12-20 20:35:14 +00:00
Jeremy Stanley
fa0c1b495c Generate HTTPS certs for Mailman sites
We're going to want Mailman 3 served over HTTPS for security
reasons, so start by generating certificates for each of the sites
we have in v2. Also collect the acme.sh logs for verification.

Change-Id: I261ae55c6bc0a414beb473abcb30f9a86c63db85
2021-12-17 22:25:22 +00:00
Zuul
ef24d3e9ce Merge "Add a domain aliases mechanism to lists.o.o" 2021-12-16 23:14:15 +00:00
Jeremy Stanley
1addce7dbc Add a domain aliases mechanism to lists.o.o
In order to be able to redirect list addresses which have moved from
one domain to another, we need a solution to alias the old addresses
to the new ones. We have simple aliases but they only match on the
local part. Add a new /etc/aliases.domain which matches full
local_part@domain addresses instead. Also collect this file in the
Mailman deployment test for ease of inspection.

Change-Id: I16f871e96792545e1a8cc8eb3834fa4eb82e31c8
2021-12-16 19:22:11 +00:00
Zuul
a1885ef992 Merge "Update limboria ircbot to bullseye" 2021-12-15 22:27:33 +00:00
Zuul
8ee2833521 Merge "Restart mailman services when testing" 2021-12-15 19:05:12 +00:00
Zuul
d328a7dd8b Merge "Collect mailman logs in deployment testing" 2021-12-15 17:46:38 +00:00
Zuul
29fbc1f078 Merge "Update matrix-eavesdrop image to bullseye" 2021-12-15 17:46:36 +00:00
Jeremy Stanley
333534fa9f Restart mailman services when testing
Mailman utilizes on-disk queues to store its actions, so doesn't act
unless its queue runners are operating. They're not started at
setup, so perform a service restart to make sure they're running in
our tests.

Change-Id: I4365f6111d4d394ed7f845660d9f342551c31e80
2021-12-15 17:42:55 +00:00
Zuul
433a744205 Merge "Copy Exim logs in system-config-run jobs" 2021-12-15 16:32:35 +00:00
Zuul
57d5e116a0 Merge "Update the accessbot image to bullseye" 2021-12-14 23:40:39 +00:00
Zuul
63fb188aa3 Merge "Update the hound image to bullseye" 2021-12-13 22:08:29 +00:00
Clark Boylan
22957c6549 Update limboria ircbot to bullseye
Spring cleaning updates of our docker images now that bullseye is out.

Change-Id: I5e4b84edd2c5a8e196659e4815c5b349c0226393
2021-12-13 09:22:17 -08:00
Clark Boylan
ed0526cd8b Update the accessbot image to bullseye
This is general spring cleaning that we are going to try and do for our
images now that bullseye is out.

Change-Id: Iad8f5b76896b88a6aafbfba0c38d0749b9d5c88f
2021-12-13 09:18:56 -08:00
Clark Boylan
b07d5eca37 Update matrix-eavesdrop image to bullseye
Just some spring cleaning now that bullseye is released.

Change-Id: I9641dae9ee7679fb45bef93e770f69d9673d75bf
2021-12-13 09:12:10 -08:00
Clark Boylan
8530ed39a1 Update the hound image to bullseye
Just some spring cleaning now that bullseye has released.

Change-Id: I1202400932860a04841d376b9f10beb89acc175c
2021-12-13 09:04:20 -08:00
Ian Wienand
5a215e0654 infra-prod: fix infra-prod-service-zookeeper soft dependency
This is a typo from the job shuffle in
I8f6150ec2f696933c93560c11fed0fd16b11bf65 -- this should be a soft
dependency.

It is currently causing periodic jobs to fail

Change-Id: Ia420e74a1d64b12b63b1697e61992c46119451dc
2021-12-13 11:01:45 +11:00
Clark Boylan
999edcc88b Remove melody
We don't need this plugin right now

Change-Id: I7b2f0d831579076d890ef8dd3bbe6e14fa1371bc
2021-12-10 10:00:41 -08:00
Jeremy Stanley
ca2455c57b Collect mailman logs in deployment testing
Get the logs from the test mailman deployments for inspection in
build results.

Change-Id: I68ea634d6048691bf14a573e66983038bc485f3c
2021-12-09 18:46:43 +00:00
Jeremy Stanley
ce18a45a16 Copy Exim logs in system-config-run jobs
It's good to be able to look at the MTA logs and see whether
anything's (attempted to be) sent, since we block SMTP egress from
these test nodes by default.

Change-Id: I02154f2b1b6cfdf1c3914d3877c80c9289057057
2021-12-09 18:46:43 +00:00