14886 Commits

Author SHA1 Message Date
Zuul
82e498fb59 Merge "Remove ask-staging* from disabled list" 2019-05-21 08:39:28 +00:00
Zuul
dd2d9b141e Merge "ask.openstack.org: switch backup user to ask01-bup" 2019-05-21 08:19:23 +00:00
Zuul
41f6aa7275 Merge "Add #starlingx to statusbot channels" 2019-05-21 08:19:21 +00:00
Ian Wienand
3fa721e541 ask.openstack.org: switch backup user to ask01-bup
The new server has new backups

Change-Id: I59ac068e8d049c1293de47979cdbe6f202fad3c6
2019-05-21 17:33:17 +10:00
Zuul
79d473614d Merge "launch.py : fix typo calling legacy dns print function" 2019-05-21 01:38:50 +00:00
Ian Wienand
73bbc6787f Bringup mirror01.dfw.rax.opendev.org
This is an initial host for testing opendev.org mirrors

Change-Id: I26b9ed1e21e2111f48bc7ecc384880c274eed213
Depends-On: https://review.opendev.org/660235
2019-05-21 11:08:30 +10:00
Ian Wienand
670107045a Create opendev mirrors
This impelements mirrors to live in the opendev.org namespace.  The
implementation is Ansible native for deployment on a Bionic node.

The hostname prefix remains the same (mirrorXX.region.provider.) but
the groups.yaml splits the opendev.org mirrors into a separate group.
The matches in the puppet group are also updated so to not run puppet
on the hosts.

The kerberos and openafs client parts do not need any updating and
works on the Bionic host.

The hosts are setup to provision certificates for themselves from
letsencrypt.  Note we've added a new handler for mirror nodes to use
that restarts apache on certificate issue/renewal.

The new "mirror" role is a port of the existing puppet mirror.pp.  It
installs apache, sets up some modules, makes some symlinks, sets up a
cleanup cron job and installs the apache vhost configuration.

The vhost configuration is also ported from the extant puppet.  It is
simplified somewhat; but the biggest change is that we have extracted
the main port 80 configuration into a macro which is applied to both
port 80 and 443; i.e. the host will have SSL support.  The other ports
are left alone for now, but can be updated in due course.

Thus we should be able to CNAME the existing mirrors to new nodes, and
any existing http access can continue.  We can update our mirror setup
scripts to point to https resources as appropriate.

Change-Id: Iec576d631dd5b02f6b9fb445ee600be060f9cf1e
2019-05-21 11:08:25 +10:00
Zuul
60f47bf05e Merge "Add testinfra master to -devel job" 2019-05-20 22:43:56 +00:00
Zuul
695a064036 Merge "Remove grafana01.openstack.org from inventory" 2019-05-20 22:23:33 +00:00
Zuul
a319e141af Merge "Add ask01.openstack.org to inventory" 2019-05-20 22:23:31 +00:00
Zuul
a4566bab0c Merge "launch.py: Fix inventory list" 2019-05-20 22:00:09 +00:00
Dirk Mueller
c43cb4a78f Properly mirror the lxc container images as well
There is convoluted code in openstack ansible CI to
fetch the file from the official mirror, which is frowned
upon for CI reliability purposes. so we have to mirror
it into AFS.

Change-Id: I84c43f8d4eb0d0ae5ca81c4f8620058a3ecc46fe
2019-05-20 15:44:10 +02:00
Zuul
2c5847dad9 Merge "Split the base playbook into services" 2019-05-20 10:04:40 +00:00
Ian Wienand
c796021bcb Add ask01.openstack.org to inventory
Change-Id: I474c0cf7bab51d2ec73a87af0a4ecbf910109c97
2019-05-20 17:56:55 +10:00
Ian Wienand
af553c45d7 ask.o.o : workaround old puppet-solr package
puppet-solr is dead upstream.  Even the un-merged pull request for
Xenial support isn't sufficient [1].

We can either get into the business of owning puppet-solr, or hack
around it.  It seems the major difference is that jetty package split
into separate jetty[8|9] packages, and puppet-solr just uses "jetty"
everywhere.

This deb, created by equivs does the following

 * pre-depends on jetty8
 * installs a symlink /etc/init.d/jetty -> jetty8
 * symlinks in the webserver directory to /usr/share/jetty

This appears to be enough to get things going.  By pre-installing it,
puppet-solr is happy enough to go on...

[1] https://github.com/vamsee/puppet-solr/pull/33

Change-Id: Ie86303caeb26634434dc4b2d0d3f1195749a277e
2019-05-20 17:25:20 +10:00
Ian Wienand
2e83c579f6 Remove ask-staging* from disabled list
These servers have been removed

Change-Id: I26ebd650866f9a71dd8b41f889878659785e4255
2019-05-20 17:25:20 +10:00
Ian Wienand
d86d1d8796 launch.py : fix typo calling legacy dns print function
Change-Id: Ia33c93320497adeffd3ea4e812f11115a6570f28
2019-05-20 13:37:07 +10:00
Ian Wienand
87d2cea6a7 launch.py: Fix inventory list
This was introduced with Ia67e65d25a1d961b619aa445303015fd577dee57

Passing "-i file1,file2,file.." makes Ansible think that the inventory
argument is a list of hostnames.  Separate out the "-i" flags so it
reads each file as desired.

Change-Id: I92c9a74de6552968da6c919074d84f2911faf4d4
2019-05-20 13:09:40 +10:00
Monty Taylor
6bc8754b87 Remove opendev k8s cluster from inventory
We're not really using/maintaining this at the moment. Before we do
put it back in production, we're likely to simply rebuild it from
scratch.

Change-Id: I469f00e90903a010f2cec45031b049556eb268a2
2019-05-19 07:36:39 -05:00
Monty Taylor
7c54c2781b Remove unreachable hosts from inventory
None of these can be reached from bridge.

Change-Id: I2f4d419a7ea9993e90dba6d25681807f98ea1db5
2019-05-19 07:36:39 -05:00
James E. Blair
8ad300927e Split the base playbook into services
This is a first step toward making smaller playbooks which can be
run by Zuul in CD.

Zuul should be able to handle missing projects now, so remove it
from the puppet_git playbook and into puppet.

Make the base playbook be merely the base roles.

Make service playbooks for each service.

Remove the run-docker job because it's covered by service jobs.

Stop testing that puppet is installed in testinfra. It's accidentally
working due to the selection of non-puppeted hosts only being on
bionic nodes and not installing puppet on bionic. Instead, we can now
rely on actually *running* puppet when it's important, such as in the
eavesdrop job. Also remove the installation of puppet on the nodes in
the base job, since it's only useful to test that a synthetic test
of installing puppet on nodes we don't use works.

Don't run remote_puppet_git on gitea for now - it's too slow. A
followup patch will rework gitea project creation to not take hours.

Change-Id: Ibb78341c2c6be28005cea73542e829d8f7cfab08
2019-05-19 07:31:00 -05:00
Jeremy Stanley
3eaf200196 Revert "Pin skopeo to unbreak skopeo+bubblewrap"
This reverts commit 0d370a285b09bd28c5b1cdfc6b89d2997f67da5d.

Fixed by https://github.com/containers/skopeo/pull/653 so safe to
merge this once a new build appears in the PPA.

Change-Id: I858eee79d084016b6b71eec46a6118d78f68cafa
2019-05-18 13:33:29 +00:00
Ian Wienand
ee4448b162 Remove puppet 3 beaker jobs
These can be removed as we don't wish to gate on puppet 3 any more.

Change-Id: I027af025ef1bdae6cd321471d2ac383711d76dea
2019-05-17 13:20:54 +10:00
Ian Wienand
829d7ef672 Remove legacy-puppet-syntax-3
We no longer need to gate against puppet 3 syntax

Change-Id: I2518eb1d85d887a98425f395b816e9c92a53282a
Needed-By: https://review.opendev.org/659696
2019-05-17 12:54:23 +10:00
Zuul
8ff026ee33 Merge "letsencrypt: use a fake CA for self-signed testing certs" 2019-05-16 23:51:19 +00:00
Zuul
33e09b7ef5 Merge "Use handlers for letsencrypt cert updates" 2019-05-16 23:51:18 +00:00
Zuul
157ad6d521 Merge "Prune docker images after docker-compose up" 2019-05-16 22:55:04 +00:00
Clark Boylan
16c255a4ca Cap ansible to <2.8 to fix testinfra
The ansible 2.8 release breaks testinfra because it does not include
paramiko anymore. Work around this by capping ansible below 2.8 until
testinfra is updated to bring paramiko along itself.

Change-Id: Ic33a08f4771207fc11af2f44104a3bcb5ec19bc5
2019-05-16 12:44:42 -07:00
Dean Troyer
1388adf6fb Add #starlingx to statusbot channels
Change-Id: I790acb9bd31908b781aa8cef3ad04e2144999706
Signed-off-by: Dean Troyer <dtroyer@gmail.com>
2019-05-16 14:40:34 -05:00
Zuul
734f4b2794 Merge "Pin skopeo to unbreak skopeo+bubblewrap" 2019-05-16 04:19:50 +00:00
Ian Wienand
d5b321b074 Handle moved puppet repos
As per [1], it seems puppet has "cleaned up" most of the packages we
are using to install.

Install the puppet-agent packages directly as puppet's archive location
is not a valid repo. With puppet 4 at least these packages should bundle
everything we need including ruby.

[1] https://groups.google.com/forum/#!msg/puppet-users/cCsGWKunBe4/OdG0T7LeDAAJ

Depends-On: https://review.opendev.org/659384
Depends-On: https://review.opendev.org/659395
Change-Id: Ie9e2b79b42f397bddd960ccdc303b536155ce123
2019-05-15 16:03:07 -07:00
Colleen Murphy
0f1c72ef13 Update ask.openstack.org to puppet 4
Change-Id: I102e42c5964fbdeabc9fef464f803b01e33e009d
2019-05-15 09:04:36 -07:00
Monty Taylor
0d370a285b Pin skopeo to unbreak skopeo+bubblewrap
Pin skopeo back to 0.1.36-1~dev~ubuntu16.04.2~ppa14 which is before
the code that changed the required capabilities, breaking the use of
skopeo from inside of bubblewrap.

Change-Id: Ibf3000d87772d02b7325315cfeed078716e0d7bf
2019-05-15 14:16:57 +00:00
Zuul
d968256e89 Merge "Update puppet-python to "fix" broken facts" 2019-05-14 23:38:48 +00:00
Zuul
4bfc2d3f65 Merge "Update lists.openstack.org to puppet 4" 2019-05-14 21:47:22 +00:00
Zuul
4f92eb85a2 Merge "Force ipv4 on vexxhost nodepool nodes" 2019-05-14 21:27:09 +00:00
Colleen Murphy
20356f1bdc Update lists.openstack.org to puppet 4
Change-Id: I90dfb0481ee2f720650b9c9a09b80151182654ec
2019-05-14 13:25:23 -07:00
Zuul
91a3ce7e4d Merge "Update zuul servers to puppet 4" 2019-05-14 20:21:03 +00:00
Clark Boylan
6176978039 Force ipv4 on vexxhost nodepool nodes
Do this in an attempt to mitigate/work around the dns resolution
problems we have had in that cloud. One thoguht is that this could be
ipv6 specific.

Change-Id: I1f9ef4a031749484d06de9427943abac4de33d29
2019-05-14 11:54:43 -07:00
Ian Wienand
1992a9c1ec letsencrypt: use a fake CA for self-signed testing certs
Production letsencrypt certificate generation creates an intermediate
chain file (ca.cer); to simulate this during the self-signed tests
generate a fake CA certifcate, and use that to sign the generated
server certificate.

Tests updated to look for all these files

Change-Id: I3990529bca7ff3c6413ed0066f9c4feaf5464b1c
2019-05-14 10:24:28 +10:00
Ian Wienand
733122f0df Use handlers for letsencrypt cert updates
This change proposes calling a handler each time a certificate is
created/updated.  The handler name is based on the name of the
certificate given in the letsencrypt_certs variable, as described in
the role documentation.

Because Ansible considers calling a handler with no listeners an error
this means each letsencrypt user will need to provide a handler.

One simple option illustrated here is just to produce a stamp file.
This can facilitate cross-playbook and even cross-orchestration-tool
communication.  For example, puppet or other ansible playbooks can
detect this stamp file and schedule their reloads, etc. then remove
the stamp file.  It is conceivable more complex listeners could be
setup via other roles, etc. should the need arise.

A test is added to make sure the stamp file is created for the
letsencrypt test hosts, which are always generating a new certificate
in the gate test.

Change-Id: I4e0609c4751643d6e0c8d9eaa38f184e0ce5452e
2019-05-14 08:14:51 +10:00
Zuul
3367358fc6 Merge "Don't gather facts in set-hostnames" 2019-05-13 14:10:28 +00:00
Monty Taylor
ad710e1a30 Don't gather facts in set-hostnames
It's a waste of energy

Change-Id: I8e456b10eec08fad4741ed51554a8c8c67fc7cbf
2019-05-13 06:28:25 +00:00
Zuul
8baf6cabd3 Merge "Rename review.openstack.org to review.opendev.org" 2019-05-12 11:46:29 +00:00
Jeremy Stanley
4cb523cdc9 Drop tools/owners.py
Now that the tools/owners.py script is a module in the
openstack_election package within the openstack/election repository,
we can stop providing a copy here.

Change-Id: I39efbad539790687646c1d76159894e9e997ff72
Depends-On: I180ef0e5ec880b46f0427c1c952b640a780b5732
2019-05-12 11:26:39 +00:00
Monty Taylor
7f0baf439e Remove unused commit-filter script
This was for cgit servers and is no longer used.

Change-Id: Iea40e04632c61b3b103769645ce0350c35ef8602
2019-05-10 11:43:23 +00:00
Ian Wienand
fc988d158b Remove grafana01.openstack.org from inventory
Replaced with grafana02.openstack.org in production

Change-Id: Ieca7b56ea9d79b5642943064e37bb99dc1b43eda
2019-05-10 09:26:53 +10:00
Zuul
bdab965d9b Merge "Mirror fedora 30 for jobs" 2019-05-09 22:27:16 +00:00
Zuul
a233ed496f Merge "Add NO_TIMEOUT for mirror update scripts" 2019-05-09 22:18:55 +00:00
Monty Taylor
e69c7b7fb9 Rename review.openstack.org to review.opendev.org
There are many references to review.openstack.org, and while the
redirect should work, we can also go ahead and fix them.

Change-Id: I28f398796a6392a3dffea1d25cfe2ae3a36a3589
2019-05-09 14:38:51 +00:00