16077 Commits

Author SHA1 Message Date
James E. Blair
496956d837 Fix whitespace in zuul-executor PPAs
Oops.

Change-Id: I3a0713299a8c6c8b71712ef7e84b10495fd5c549
2020-05-20 16:24:51 -07:00
James E. Blair
9662805937 Correct the test gearman certs
The existing test gearman cert+key combos were mismatched and therefore
invalid.  This replaces them with newly generated test data, and moves
them into the test private hostvar files where the production private
data are now housed.

This removes the public production data as well; those certs are now
in the private hostvar files.

Change-Id: I6d7e12e2548f4c777854b8738c98f621bd10ad00
2020-05-20 14:54:20 -07:00
Clark Boylan
eb22e01f31 Add support for multiple jvbs behind meetpad
The jitsi video bridge (jvb) appears to be the main component we'll need
to scale up to handle more users on meetpad. Start preliminary
ansiblification of scale out jvb hosts.

Note this requires each new jvb to run on a separate host as the jvb
docker images seem to rely on $HOSTNAME to uniquely identify each jvb.

Change-Id: If6d055b6ec163d4a9d912bee9a9912f5a7b58125
2020-05-20 13:41:30 -07:00
James E. Blair
085856e318 Add iptables_extra_allowed_groups
This adds a new variable for the iptables role that allows us to
indicate all members of an ansible inventory group should have
iptables rules added.

It also removes the unused zuul-executor-opendev group, and some
unused variables related to the snmp rule.

Also, collect the generated iptables rules for debugging.

Change-Id: I48746a6527848a45a4debf62fd833527cc392398
Depends-On: https://review.opendev.org/728952
2020-05-20 13:18:29 -07:00
James E. Blair
09935ff328 Run Zuul as the zuuld user
This avoids the conflict with the zuul user (1000) on the test
nodes.  The executor will continue to use the default username
of 'zuul' as the ansible_user in the inventory.

This change also touches the zk and nodepool deployment to use
variables for the usernames and uids to make changes like this
easier.  No changes are intended there.

Change-Id: Ib8cef6b7889b23ddc65a07bcba29c21a36e3dcb5
2020-05-20 13:17:28 -07:00
James E. Blair
b173fcb1d9 Vendor the apt repo gpg keys used for Zuul
We use several PPAs on the Zuul servers, and today the Ubuntu keyring
servers are frequently failing.  Rather than rely on them, store the
GPG keys in this repo and install the files "manually" rather than
using the apt_repo module.

Change-Id: I009a1a38d3a5864a8d5b0d8f8be24a83d1924292
2020-05-20 13:17:09 -07:00
James E. Blair
7a63dad5c1 Save zuul and nodepool logs from gate test jobs
Let's save our debug logs so we can better observe the system in
the gate.

Change-Id: Ic80b646e0407d27e43cdb10cb573551999dd01d4
2020-05-20 13:17:08 -07:00
Zuul
4f0a82b125 Merge "Listen on Quay Registry Mirror Ports" 2020-05-20 19:17:50 +00:00
Zuul
192b77ade8 Merge "Open mirror ssl ports externally" 2020-05-20 18:37:16 +00:00
Zuul
2dff1e482d Merge "Use ensure-nodejs in Gerrit deployment testing" 2020-05-20 15:45:59 +00:00
Clark Boylan
c0ac38c2c9 Listen on Quay Registry Mirror Ports
We need to tell apache to listen on the ports used by the Quay Registry
Mirror. Without this we aren't actually able to provide connections to
this vhost.

Add testing to ensure this is working in a simple manner.

Change-Id: I28bdb7aeb9c3252c6319658acaa530a7d7c25a72
2020-05-19 16:47:51 -07:00
Clark Boylan
d8f01de682 Open mirror ssl ports externally
This was missed in an earlier change where we enabled these vhosts.
Testing worked because testing was communicating to localhost and not
the public ip address.

This has been addressed as well.

Change-Id: I2d91aea466f1b587780a452cfe8e1396515930ed
2020-05-19 16:47:22 -07:00
Zuul
728f8a9ee5 Merge "Enable ssl on all mirror vhosts" 2020-05-19 21:38:12 +00:00
Zuul
58e52a7047 Merge "Add missing HTTPS ports in ssldomains file" 2020-05-19 21:28:07 +00:00
Zuul
10530e2363 Merge "Add OpenEdge CI mirror to Cacti config" 2020-05-19 21:24:46 +00:00
Zuul
dd6e5fdfbb Merge "Run multi-node-hosts-file in run-base-pre" 2020-05-19 20:34:07 +00:00
Jeremy Stanley
f5f715008c Add missing HTTPS ports in ssldomains file
Some entries were added to the ssldomains list lacking a port
number. Add the HTTPS port to them.

Change-Id: I6bea5cbabb63ada9d817725e652157ccbdce7929
2020-05-19 20:32:46 +00:00
Jeremy Stanley
448a9b3388 Add OpenEdge CI mirror to Cacti config
This was missed when the mirror server was initially built.

Change-Id: I838e2ab11d2db308d947e10adad8f1611d051f5c
2020-05-19 20:32:44 +00:00
Jeremy Stanley
dd697fc313 Use ensure-nodejs in Gerrit deployment testing
The install-nodejs role in zuul-jobs has been replaced by
ensure-nodejs, so we should use the new thing if we want our tests
running again.

Change-Id: I196814b616d3b332b2c1d397097c01b5bb0d2aac
2020-05-19 19:16:26 +00:00
Clark Boylan
79ff2afb87 Enable ssl on all mirror vhosts
Previously we had enabled SSL on our main vhost for the mirrors. Do
similar for all of the proxy cache vhosts for docker and other external
resources.

As part of this change we improve the testing to ensure that the new
vhosts are working as expected. One testing specific change to note is
the testinfra node names did not match our existing system-config-run
job nodenames. This has been corrected.

Additionally RHRegistryMirror and QuayMirror may not be working and
fixing those is left as a followup.

Change-Id: I9dbbd4080c3a2cce4acc39d63244f7a645503553
2020-05-19 11:52:20 -07:00
Zuul
4589392550 Merge "Set connection limits on mirror apache workers" 2020-05-18 19:18:01 +00:00
Zuul
8945cdb26a Merge "Remove puppet mirror support" 2020-05-18 18:50:50 +00:00
Zuul
69ad27d9ab Merge "Disable jitsi watermark in jitsi conferences" 2020-05-18 16:34:06 +00:00
Clark Boylan
c6ed48a6f4 Disable jitsi watermark in jitsi conferences
Disable the jitsi logo watermark which is displayed in the top left
portion of the jitsi conference toolbar. The reason for this is it seems
to cause rendering issues with the etherpads that are loaded in meetpad.
Even without the rendering issues it covers the beginning portion of
etherpads which is annoying.

To do this we copy over the interface config and edit the two variables
related to displaying the watermark. All other values are kept
consistent with the defaults in the docker image.

Change-Id: I1e363d780083894fc53a030349fbc36567891271
2020-05-16 12:00:02 -07:00
Ian Wienand
45201f3d66 Remove puppet mirror support
Remove the separate "mirror_opendev" group and rename it to just
"mirror".  Update various parts to reflect that change.

We no longer deploy any mirror hosts with puppet, remove the various
configuration files.

Depends-On: https://review.opendev.org/728345
Change-Id: Ia982fe9cb4357447989664f033df976b528aaf84
2020-05-16 10:14:25 +10:00
Ian Wienand
6acc2dca98 Remove limestone openstack.org mirror
This mirror has been replaced by the opendev.org version

Change-Id: I86b706adec9dd6ce02baed87c0404f11ab1d4104
Depends-On: https://review.opendev.org/728318
2020-05-16 10:14:25 +10:00
Ian Wienand
0e2059267d Remove vexxhost openstack.org mirrors
Hosts are no longer active

Change-Id: I0247323ad2df70c8dd4b165d8bb307c10f947138
Depends-On: https://review.opendev.org/728310
2020-05-16 10:14:25 +10:00
Ian Wienand
4233b79e31 Add limestone opendev.org server
This is to replace the puppet managed openstack.org server

Change-Id: I0e3586befd922cb56d1a0ec9c9cb650add9b225d
Depends-On: https://review.opendev.org/728314
2020-05-16 10:14:25 +10:00
Ian Wienand
a864212b1b Add vexxhost opendev.org mirrors
These are to replace the puppet-based openstack.org mirrors

Depends-On: https://review.opendev.org/728308
Change-Id: Ibdce99daa514fb445f1f8389e7c052ee151057ea
2020-05-16 10:14:25 +10:00
Ian Wienand
da6d1cbd06 Remove linaro-london cloud
This cloud is no longer used

Change-Id: I14ab277b3877f6674ec3172c06a39f383e76a1d0
Depends-On: https://review.opendev.org/728332
2020-05-16 10:14:09 +10:00
Clark Boylan
a83a763644 Add meetpad to cacti and ssl certcheck
People are starting to use this service so having performance metrics
over time is a good thing. We also want to avoid having our cert expire
unexpectedly.

Change-Id: I744b3e68f8f483b36c0d8ecb6f6f46a484a3577a
2020-05-15 13:51:33 -07:00
Ian Wienand
3a47fb5b8f Remove citycloud
This provider has been retired for now

Change-Id: Ia5cdda34cb50eebb2d7920eb3010db6ab7d89fd6
Depends-On: https://review.opendev.org/727902
2020-05-15 13:11:28 +10:00
Ian Wienand
38f36c36c9 Remove iad.rax openstack.org mirror
This host is no longer used

Change-Id: If626d851c775ae199d968f0eb0c22f53f753316f
Depends-On: https://review.opendev.org/727917
2020-05-15 13:11:02 +10:00
Zuul
f2bef7ae1a Merge "Add infra-root-keys-2020-05-13 to rotate older ssh keys" 2020-05-15 00:29:26 +00:00
Zuul
398e3e4335 Merge "Remove mirror01.ord.rax.openstack.org" 2020-05-15 00:20:52 +00:00
Zuul
525717c427 Merge "Remove mirror02.dfw.rax.openstack.org" 2020-05-15 00:12:59 +00:00
Ian Wienand
a751cab84d Replace mirror.iad.rax.opendev.org host
This is a rebuilt host for this mirror, the old one has been removed.

Depends-On: https://review.opendev.org/727899
Change-Id: Icaf4ad1d88a40c0abdfe1e87386b3c557de062a1
2020-05-14 11:13:51 +10:00
Ian Wienand
05c8f2b7ae Remove mirror01.ord.rax.openstack.org
This old mirror host is no longer used and replaced by the opendev.org
server, remove it.

Change-Id: I10abbd89728330217d60bb1c300a17560c15dc75
2020-05-14 10:28:42 +10:00
Ian Wienand
80e4b617f1 Remove mirror02.dfw.rax.openstack.org
Replace by opendev mirror

Change-Id: Id5fc956421948c405d5675a746b5c4258905ac74
Depends-On: https://review.opendev.org/690757
2020-05-14 10:02:42 +10:00
Zuul
5e6732f6a4 Merge "Add focal testing for mirror nodes" 2020-05-13 23:55:12 +00:00
Zuul
c4dffb8b9e Merge "Update to Ansible 2.9.8" 2020-05-13 23:53:11 +00:00
Clark Boylan
33c744d923 Set connection limits on mirror apache workers
We've noticed that our mirrors will semi regularly have problems due to
old stale works. For example using old ssl certs or having connection
problems to round robin backend services. In all cases restarting the
service (killing old workers) seems to fix things. Try to force this to
automatically happen by setting a reasonable connection limit per worker
before we recycle them.

Change-Id: Ic377f48d1a5a3eecbcb183327c9255134c4364ab
2020-05-13 13:46:38 -07:00
Clark Boylan
c4a6ba9c03 Add infra-root-keys-2020-05-13 to rotate older ssh keys
Due to the way OpenStack Ansible and Nova work we can't update the value
for our ssh keys entries in nova. We need to create a new key then
switch everything over to that. Unfortunately, we've had a few people
request their keys be removed and a few others that haven't been active
since the last rotation. That means now is a good time to do a rotation
to clean up the set of valid keys.

Change-Id: Idb7c5eaa546096784e428b3cde61581dcc9050a2
2020-05-13 13:15:43 -07:00
Clark Boylan
f857b81295 Remove old status server from inventory
This server was removed everywhere but our inventory. I checked it
doesn't show up in a nova listing. Remove it from here so we can remove
it from the emergency.yaml file as well.

Change-Id: I154f7c16166e6ac6610c0ea02e5d59fa7baa25c0
2020-05-13 10:27:12 -07:00
James E. Blair
6a53d47050 Exclude some regex matches in jitsi-meet web
If we match these explicit paths, don't allow the regex matches to
run.  Otherwise, we might redirect "/http-bind" to "/" which doesn't
work.

Also, restore the default meet.conf to the upstream; we're not
intending to carry any local changes to the rootfs.

Change-Id: I119d8c223291d79bd5fe0977264dec4bae126280
2020-05-13 07:40:18 -07:00
Zuul
60b696da09 Merge "Replace OVH CI mirrors" 2020-05-13 01:12:00 +00:00
Jeremy Stanley
c3ad4ee822 Update docs.airshipit.org functional test
The string previously expected in the docs.airshipit.org site is no
longer used there, so pick another which will get the
system-config-run-static job passing again.

Change-Id: I8bf0da25408412dc80ff8cb6d26c185234916941
2020-05-12 21:48:55 +00:00
Ian Wienand
7b8b788ce2 Add focal testing for mirror nodes
Change-Id: I64de9a61c5044b93f6ce7e2d31cf51d78fd4ec16
2020-05-13 05:32:54 +10:00
Ian Wienand
52943f7094 Update to Ansible 2.9.8
This has an important fix for the systemd output that has changed on
Ubuntu Focal

Change-Id: I9592528c557914cb142229dadaa84ee1742c7b86
2020-05-13 05:32:54 +10:00
Jeremy Stanley
31acca81d7 Replace OVH CI mirrors
New opendev.org CI mirrors for OVH regions. The old BHS1 mirror was
in the openstack.org domain, so is added new. There was an old GRA1
mirror in the opendev.org domain, so remote it and increment the
ordinal in its short hostname to avoid a collision in the inventory
cache.

This is being done to switch to un-billed flavors in this provider,
to simplify internal billing for their donation of resources.

Change-Id: I05770856b5704aa438ed6bc54ec42ba9efb5cd2a
2020-05-12 19:10:21 +00:00