Commit Graph

17679 Commits (fa0c1b495c02515b409f24406e4dde4f6e6ef824)

Author SHA1 Message Date
Jeremy Stanley fa0c1b495c Generate HTTPS certs for Mailman sites
We're going to want Mailman 3 served over HTTPS for security
reasons, so start by generating certificates for each of the sites
we have in v2. Also collect the logs for verification.

Change-Id: I261ae55c6bc0a414beb473abcb30f9a86c63db85
2021-12-17 22:25:22 +00:00
Zuul 6324944318 Merge "Switch router addresses for review02 to global" 2021-12-17 17:16:15 +00:00
Zuul c4f815e885 Merge "Set CacheMaxExpire to 1h" 2021-12-17 17:00:58 +00:00
Dr. Jens Harbott cd7624fa2b Switch router addresses for review02 to global
Vexxhost wants to change the routers for their IPv6 setup, which will
change their link-local addresses. Change our setup to use the global
addresses instead, which will stick.

Change-Id: I45c6a3b776645294a688329c60949c0c3c4529a5
2021-12-17 16:32:59 +01:00
Dr. Jens Harbott 84278365d0 Set CacheMaxExpire to 1h
We want to limit the time we remember possibly broken index responses
which we sometimes receive from the pypi CDN. We cannot set this per
location, so this is a comprise between reducing the impact of bad eggs
in the cache and trying not to throw out the good eggs too fast.

Change-Id: If88f10cb7e3cebfa9c37a71d284d513f25b8bb52
2021-12-17 11:40:51 +01:00
Zuul 8133805f29 Merge "Create an OpenInfra Foundation staff ML" 2021-12-16 23:18:07 +00:00
Zuul ef24d3e9ce Merge "Add a domain aliases mechanism to lists.o.o" 2021-12-16 23:14:15 +00:00
Jeremy Stanley f906b06555 Create an OpenInfra Foundation staff ML
This is a new mailing list into which the current staff ML from the site will be manually migrated. The existing one
is not included in our current configuration anyway, but a followup
change will set up an appropriate forward for its old address once
migration is complete.

Change-Id: I15f47d210e38a8f04925ffba27e44b2ad5e97dd5
2021-12-16 19:22:16 +00:00
Jeremy Stanley 1addce7dbc Add a domain aliases mechanism to lists.o.o
In order to be able to redirect list addresses which have moved from
one domain to another, we need a solution to alias the old addresses
to the new ones. We have simple aliases but they only match on the
local part. Add a new /etc/aliases.domain which matches full
local_part@domain addresses instead. Also collect this file in the
Mailman deployment test for ease of inspection.

Change-Id: I16f871e96792545e1a8cc8eb3834fa4eb82e31c8
2021-12-16 19:22:11 +00:00
Zuul 3912cf21c6 Merge "Add openstack-venus channel in statusbot" 2021-12-16 09:54:24 +00:00
Zuul 5e70cb6aed Merge "zuul-*: use multiline formatter" 2021-12-16 00:06:45 +00:00
Zuul 7b39ec46e2 Merge "Fix command for setting the entry message for IRC channel" 2021-12-15 22:27:36 +00:00
Zuul a1885ef992 Merge "Update limboria ircbot to bullseye" 2021-12-15 22:27:33 +00:00
Ghanshyam Mann 4f2bbe301c Fix command for setting the entry message for IRC channel
In OFTC, entery message is set via ``entrymsg`` command,
correcting it in doc.

<ChanServ> *** SET Help ***
URL: Set the channel's homepage.
EMAIL: Sets the channel's e-mail address.
ENTRYMSG: Sets the channel greeting.

Change-Id: I2e436015641ab78c5b509b4b4ca35e1088c3376f
2021-12-15 16:02:40 -06:00
Ghanshyam Mann 9dde035e8a Add openstack-venus channel in statusbot
openstack/venus project is newly added
and channel is being added in project-config
accessbot by depends on patch.

Change-Id: Ibf98e54850f65968710a5161d77d3d0880642f38
2021-12-15 15:29:44 -06:00
Zuul 2863b5a509 Merge "Use newlist's automate option" 2021-12-15 19:09:57 +00:00
Zuul 8ee2833521 Merge "Restart mailman services when testing" 2021-12-15 19:05:12 +00:00
Zuul 3c7849c44a Merge "Add "mailman" meta-list to" 2021-12-15 18:31:13 +00:00
Zuul a5af748939 Merge "Make sure /usr/bin/python is present for mailman" 2021-12-15 18:27:10 +00:00
Zuul d328a7dd8b Merge "Collect mailman logs in deployment testing" 2021-12-15 17:46:38 +00:00
Zuul 29fbc1f078 Merge "Update matrix-eavesdrop image to bullseye" 2021-12-15 17:46:36 +00:00
Jeremy Stanley 759e285184 Use newlist's automate option
It appears that simply setting stdin to an empty string is
insufficient to make newlist calls from Ansible correctly look like
they're coming from a non-interactive shell. As it turns out, newer
versions of the command include a -a (--automate) option which does
exactly what we want: sends list admin notifications on creation
without prompting for manual confirmation.

Drop the test-time addition of -q to quell listadmin notifications,
as we now block outbound 25/tcp from nodes in our deploy tests. This
has repeatedly exposed a testing gap, where the behavior in
production was broken because of newlist processes hanging awaiting
user input even though we never experienced it in testing due to the
-q addition there.

Change-Id: I550ea802929235d55750c4d99c7d9beec28260f0
2021-12-15 17:42:58 +00:00
Jeremy Stanley 333534fa9f Restart mailman services when testing
Mailman utilizes on-disk queues to store its actions, so doesn't act
unless its queue runners are operating. They're not started at
setup, so perform a service restart to make sure they're running in
our tests.

Change-Id: I4365f6111d4d394ed7f845660d9f342551c31e80
2021-12-15 17:42:55 +00:00
Zuul 433a744205 Merge "Copy Exim logs in system-config-run jobs" 2021-12-15 16:32:35 +00:00
Zuul 57d5e116a0 Merge "Update the accessbot image to bullseye" 2021-12-14 23:40:39 +00:00
Jeremy Stanley 196b081159 Add "mailman" meta-list to
Mailman uses a (usually hidden) mailing list named "mailman" to
handle things like password reminders and certain other sorts of
notifications. We have one in the configuration for all the sites on but not on, even though
the production server has one. Not creating this list will cause
the services to fail to start, and since we want to test restarting
them in an upcoming change, add the missing entry (it will be a
no-op in production anyway).

Change-Id: If06d9d060e40055f95c1df337eb6f32c6064a89f
2021-12-14 21:04:41 +00:00
Zuul b044cba65a Merge "Block outbound SMTP connections from test jobs" 2021-12-14 20:46:12 +00:00
Zuul 63fb188aa3 Merge "Update the hound image to bullseye" 2021-12-13 22:08:29 +00:00
Clark Boylan 22957c6549 Update limboria ircbot to bullseye
Spring cleaning updates of our docker images now that bullseye is out.

Change-Id: I5e4b84edd2c5a8e196659e4815c5b349c0226393
2021-12-13 09:22:17 -08:00
Clark Boylan ed0526cd8b Update the accessbot image to bullseye
This is general spring cleaning that we are going to try and do for our
images now that bullseye is out.

Change-Id: Iad8f5b76896b88a6aafbfba0c38d0749b9d5c88f
2021-12-13 09:18:56 -08:00
Clark Boylan b07d5eca37 Update matrix-eavesdrop image to bullseye
Just some spring cleaning now that bullseye is released.

Change-Id: I9641dae9ee7679fb45bef93e770f69d9673d75bf
2021-12-13 09:12:10 -08:00
Clark Boylan 8530ed39a1 Update the hound image to bullseye
Just some spring cleaning now that bullseye has released.

Change-Id: I1202400932860a04841d376b9f10beb89acc175c
2021-12-13 09:04:20 -08:00
Ian Wienand 2e261fdc42 zuul-*: use multiline formatter
Zuul change I6d7e7e7a9e19d46a744f9ffac8d532fc6b4bba01 introduced a
multi-line formatter that makes exceptions and other multi-line output
much easier to follow in the logs.  Use it here for the simple
formatter in the production Zuul deployment.

Change-Id: I9a8aad8a90f5f4080cdb872d0ed65697a180f57c
2021-12-13 14:54:16 +11:00
Ian Wienand 5a215e0654 infra-prod: fix infra-prod-service-zookeeper soft dependency
This is a typo from the job shuffle in
I8f6150ec2f696933c93560c11fed0fd16b11bf65 -- this should be a soft

It is currently causing periodic jobs to fail

Change-Id: Ia420e74a1d64b12b63b1697e61992c46119451dc
2021-12-13 11:01:45 +11:00
Jeremy Stanley 4b173eaddb No lookups in Gerrit's log4j2 message formatting
This is a safety net in case Gerrit or one of its plugins is using
log4j2 in unsafe ways.

Change-Id: I9d0a05fdad379a1e47f88cc6faa9425614f6515b
2021-12-10 20:07:36 +00:00
Clark Boylan 999edcc88b Remove melody
We don't need this plugin right now

Change-Id: I7b2f0d831579076d890ef8dd3bbe6e14fa1371bc
2021-12-10 10:00:41 -08:00
Jeremy Stanley 9c7f4fad46 Make sure /usr/bin/python is present for mailman
Mailman v2.1 is still a Python2-only application, and expects
/usr/bin/python to be present. On Ubuntu Focal, there is no such
symlink provided by the Python 2.7 packages, and an extra
python-is-python2 transitional package is used to explicitly create
it in cases where that's required.

Change-Id: I37ca2bd7011afdb3b97e34cdc24ff455b9fb0498
2021-12-09 18:46:43 +00:00
Jeremy Stanley ca2455c57b Collect mailman logs in deployment testing
Get the logs from the test mailman deployments for inspection in
build results.

Change-Id: I68ea634d6048691bf14a573e66983038bc485f3c
2021-12-09 18:46:43 +00:00
Jeremy Stanley ce18a45a16 Copy Exim logs in system-config-run jobs
It's good to be able to look at the MTA logs and see whether
anything's (attempted to be) sent, since we block SMTP egress from
these test nodes by default.

Change-Id: I02154f2b1b6cfdf1c3914d3877c80c9289057057
2021-12-09 18:46:43 +00:00
Jeremy Stanley e2dbda1bec Block outbound SMTP connections from test jobs
Our deployment tests don't need to send E-mail messages. More to the
point, they may perform actions which would like to send E-mail
messages. Make sure, at the network level, they'll be prevented from
doing so. Also allow all connections to egress from the loopback
interface, so that services like mailman can connect to the Exim MTA
on localhost.

Add new rolevars for egress rules to support this, and also fix up
some missing related vars in the iptables role's documentation.

Change-Id: If4acd2d3d543933ed1e00156cc83fe3a270612bd
2021-12-09 18:46:38 +00:00
Zuul 0645a481d0 Merge "Switch lodgeit to run under a dedicated user" 2021-12-09 16:58:07 +00:00
Zuul 5df37e6a09 Merge "Add keycloak auth config to Zuul" 2021-12-09 00:51:07 +00:00
Zuul e758d24fa7 Merge "Rename install-ansible to bootstrap-bridge" 2021-12-08 20:49:53 +00:00
James E. Blair dbc69021e2 Add zuul-client config to schedulers
This adds a zuul-client config file as well as a convenience script
to execute the docker container to the schedulers.

Change-Id: Ief167c6b7f0407f5eaebecde552e8d91eb3d4ab9
2021-12-07 14:26:29 -08:00
James E. Blair 781152332d Add keycloak auth config to Zuul
This adds a keycloak realm to the Zuul auth config, so that we can
log into the zuul web ui with our test realm in keycloak.

Change-Id: Iec3777a6ea1cba0e108c7e44067d69b61cbb34a7
2021-12-07 14:19:37 -08:00
Zuul d991865e58 Merge "Update bridge playbook match" 2021-12-07 20:57:07 +00:00
Zuul 5a2f1c7037 Merge "Add local auth provider to zuul" 2021-12-07 17:54:57 +00:00
Ian Wienand 73a9acc7ad Rename install-ansible to bootstrap-bridge
This used to be called "bridge", but was then renamed with
Ia7c8dd0e32b2c4aaa674061037be5ab66d9a3581 to install-ansible to be

It is true that this is installing Ansible, but as part of our
reworking for parallel jobs this is the also the synchronisation point
where we should be deploying the system-config code to run for the

Thus naming this "boostrap-bridge" should hopefully be clearer again
about what's going on.

I've added a note to the job calling out it's difference to the
infra-prod-service-bridge job to hopefully also avoid some of the
inital confusion.

Change-Id: I4db1c883f237de5986edb4dc4c64860390cc8e22
2021-12-07 16:24:53 +11:00
Ian Wienand 362d8fa147 Update bridge playbook match
This playbook was renamed "install-ansible.yaml" with

We want all jobs to match on this; it will make them run if we update
the ansible version on the bastion host, bridge.

Change-Id: Id38fc39f8f6b4d8f532eb9796259e8f4bf18d861
2021-12-07 16:24:41 +11:00
Zuul 82edab1d39 Merge "Add comments to manage-projects about project-config syncing" 2021-12-07 00:39:57 +00:00