112 Commits

Author SHA1 Message Date
Monty Taylor
5e6aa5e70d Use python3 for ansible
We get deprecation warnings from ansible about use
of python2 on xenial hosts. Rather than setting
ansible_python_interpreter to python3 on a host by
host basis, set it globally to python3.

Set it to python for the one host that's too old,
refstack.openstack.org, which is running on trusty
which only has python3.4.

Change-Id: I4965d950c13efad80d72912911bc7099e9da1659
2020-04-28 11:54:15 -05:00
Zuul
d3360a7d60 Merge "Remove two unused ansible vars files" 2020-04-24 17:17:33 +00:00
Zuul
b21a8e58cf Merge "Run Zuul using Ansible and Containers" 2020-04-24 16:31:42 +00:00
Monty Taylor
275ccd9b80 Remove two unused ansible vars files
Change-Id: I415b709399df28d4da55b8fa0a204110729e70a8
2020-04-24 10:44:32 -05:00
Monty Taylor
f0b77485ec Run Zuul using Ansible and Containers
Zuul is publishing lovely container images, so we should
go ahead and start using them.

We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.

Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.

Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
2020-04-24 09:18:44 -05:00
Monty Taylor
99aa528c83 Stop logging puppet to syslog
We run puppet with ansible now pretty much all the time. It's not
helpful for the puppet output to go to syslog on the remote host.
What's more helpful is for it to come back to the stdout in the
ansible playbook so that we can see it.

Also turn off ansi color from the output.

Depends-On: https://review.opendev.org/721732
Change-Id: I604081d5400bd53b8dda5a3a7685323c1443991b
2020-04-23 19:38:51 +00:00
Monty Taylor
9fd2135a46 Split eavesdrop into its own playbook
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.

Add the ability to override the keys for the zuul user.

Remove openstack_project::server, it doesn't do anything.

Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.

Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
2020-04-23 14:34:28 -05:00
Monty Taylor
ebae022d07 Use project-config from zuul instead of direct clones
We use project-config for gerrit, gitea and nodepool config. That's
cool, because can clone that from zuul too and make sure that each
prod run we're doing runs with the contents of the patch in question.

Introduce a flag file that can be touched in /home/zuulcd that will
block zuul from running prod playbooks. By default, if the file is
there, zuul will wait for an hour before giving up.

Rename zuulcd to zuul

To better align prod and test, name the zuul user zuul.

Change-Id: I83c38c9c430218059579f3763e02d6b9f40c7b89
2020-04-15 12:29:33 -05:00
Monty Taylor
211a9950f5 Add zone keys to zuulcd user
We want to trigger nameserver updates when we merge patches
to zone files.

The zuul zone repo is currently managed by infra-core. We need to
make an improvement to zuul before we can offload core role there
to the zuul-maint team.

Change-Id: I6192f2499465844ccf2a1f903a8897458814da5d
2020-04-02 08:14:45 -05:00
Zuul
ce3a064133 Merge "Add meetpad server" 2020-03-27 14:44:30 +00:00
David Shrewsbury
b0e2df07b4 Remove shrews from infra-root
Change-Id: I55d9670f216fb6d36be8ec080fcc02e40bf83a68
2020-03-26 13:38:02 -04:00
James E. Blair
8b093dacd5 Add meetpad server
Depends-On: https://review.opendev.org/714189
Change-Id: I5863aaa805a18f9085ee01c3205b0f9ad602922d
2020-03-25 07:44:24 -07:00
Zuul
1189e2df0d Merge "zookeeper: open firewall port to nb04" 2020-03-19 01:31:12 +00:00
Zuul
bf125924b9 Merge "nodepool-builder container: give nodepool group acess to config" 2020-03-19 01:31:10 +00:00
Zuul
927072831b Merge "Fix URLs after OpenDev rename" 2020-03-19 01:19:16 +00:00
Ian Wienand
abec70614b zookeeper: open firewall port to nb04
We removed nb01 with I18ab9834ad4da201774e0abef56f618cd7839d36 and
replaced it with nb04; open the firewall for it.

Change-Id: I7138ee8744d978388b95e35ddd767cc97a5f5a87
2020-03-19 10:50:05 +11:00
Ian Wienand
34ec808a69 nodepool-builder container: give nodepool group acess to config
The container can't read /etc/openstack config files; give group
access so it can read them only.

Change-Id: I6f8f00a0a26995c56a147dd8f5c0b89672f840b4
2020-03-19 10:37:17 +11:00
Andreas Jaeger
173118e471 Fix URLs after OpenDev rename
As part of OpenDev rename, a lot of links were changed.
A couple of URLs point to old locations, update them.

This list was done while grepping for "openstack-infra" and fixing
locations that are wrong.

Change-Id: I313d76284bb549f1b2c636ce17fa662c233c0af9
2020-03-18 18:23:17 +01:00
Ian Wienand
b967495dc3 nodepool-builder: put container configs in /etc
Currently we deploy the openstacksdk config into ~nodepool/.config on
the container, and then map this directory back to /etc/openstack in
the docker-compose.  The config-file still hard-codes the
limestone.pem file to ~nodepool/.config.

Switch the nodepool-builder_opendev group to install to
/etc/openstack, and update the nodepool config file template to use
the configured directory for the .pem path.

Also update the testing paths.

Story: #2007407
Task: #39015
Change-Id: I9ca77927046e2b2e3cee9a642d0bc566e3871515
2020-03-17 07:37:00 +11:00
Ian Wienand
dbe0bf1ee6 Add nb01.opendev.org
This configures an opendev nodepool-builder

Change-Id: Id8603d9d7caaac0a1ab935e1c7c80d32b02ae23e
Depends-On: https://review.opendev.org/693118
2020-03-11 09:16:31 +11:00
Ian Wienand
281425a44d Add initial Ansible for nodepool hosts
This is a start at ansible-deployed nodepool environments.

We rename the minimal-nodepool element to nodepool-base-legacy, and
keep running that for the old nodes.

The groups are updated so that only the .openstack.org hosts will run
puppet.  Essentially they should remain unchanged.

We start a nodepool-base element that will replace the current
puppet-<openstackci|nodepool> deployment parts.  For step one, this
grabs project-config and links in the elements and config file.

A testing host is added for gate testing which should trigger these
roles.  This will build into a full deployment test of the builder
container.

Change-Id: If0eb9f02763535bf200062c51a8a0f8793b1e1aa
Depends-On: https://review.opendev.org/#/c/710700/
2020-03-06 14:02:52 +11:00
Ian Wienand
d1fa8c6482 Allow mirror-update.opendev.org to send stats
I forgot this in some of the prior changes that moved afsmon and
afs-release.py to this host, and those jobs send stats.

Change-Id: Ifacf69e7fef5b54a03d43272e9cc01b6fbe8e845
2020-02-17 17:58:49 +11:00
James E. Blair
53338653fd Update zuul-ci.org certs
We have a single vhost for zuul-ci.org and zuulci.org, so we should
request a cert with all 4 hostnames.

We also have a separate vhost to handle the git.zuul-ci.org redirect;
add a cert request for that so we can manage it with LE.

Change-Id: Ia2ba3d3ad4f5ab0356ede371d94af3c77a89eda1
2020-01-07 14:35:25 -08:00
James E. Blair
6288a3c016 Get letsencrypt certs for zuul-ci.org
Change-Id: Ieb0c6d02c11a660c063536206e3f9210796007b8
2020-01-06 08:56:47 -08:00
Ian Wienand
f57154f91b vos-release: have separate user
I was trying to simplify things by having a restricted shell script
run by root.  However, our base-setup called my bluff as we also need
to setup sshd to allow remote root logins from specific addresses.

It's looking easier to create a new user, and give it sudo permissions
to run the vos release script.

Change-Id: If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5
2019-11-21 12:03:45 +11:00
Ian Wienand
3153f27c24 vos-release: fix key sourcing; disable exclusive key
I wasn't correctly sourcing the key; it has to come from hostvars as
it is in a different play on different hosts.  This fixes it.

We also need to not have the base roles overwrite the authorized_keys
file each time.  The key we provision can only run a limited script
that wraps "vos release".

Unfortunately our gitops falls down a bit here because we don't have
full testing for the AFS servers; put this on the todo list :) I have
run this manually for testing.

Change-Id: I0995434bde7e43082c01daa331c4b8b268d9b4bc
2019-11-21 07:28:49 +11:00
Kevin Carter
8b0877cb68
Add proxy for quay
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.

Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-09-19 16:49:20 -05:00
James E. Blair
48cafd19f8 Add LE cert for logs.opendev.org to static
This can be used in an apache vhost later, but should be fine to
merge now.

Depends-On: https://review.opendev.org/673902
Change-Id: Ic2cb7585433351ec1bdabd88915fa1ca07da44e7
2019-07-31 13:00:50 -07:00
Jeremy Stanley
6631b899c5 Put gitea07 and gitea08 back into service
Add the gitea07.opendev.org and gitea08.opendev.org servers into the
haproxy pools now that they've been seeded with current data. Remove
the create repos task disable list entries for them as well.

Change-Id: I69390e6a32b01cc1713839f326fa930c376282af
2019-07-29 23:35:36 +00:00
Jeremy Stanley
56a0b08aa5 Swap gitea05 into service and bring down 07 and 08
Add the gitea05.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 05 to 07 and 08, and remove 07 and 08 from the
Ansible inventory and comment them out in the haproxy pools in
preparation for replacement.

To the casual observer it may appear gitea06 is being skipped, but
it was replaced first out of sequence due to filesystem corruption
during the PTG. The increased performance of the 75% of the nodes
which have already been replaced means we can get by doing the final
25% at the same time (so two servers at once).

Change-Id: Ia49157c16582b7ed0dbef3eb9d07bf7f1d4450b9
2019-07-29 16:56:39 +00:00
Jeremy Stanley
79c86cfe3d Swap gitea04 into service and bring down gitea05
Add the gitea04.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 04 to 05, and remove 05 from the Ansible inventory
and comment it out in the haproxy pools in preparation for
replacement.

Change-Id: I4cd1fef399e527771a26efee8a39952694f3ce6b
2019-07-28 12:15:41 +00:00
Jeremy Stanley
0256ba5219 Swap gitea03 into service and bring down gitea04
Add the gitea03.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 03 to 04, and remove 04 from the Ansible inventory
and comment it out in the haproxy pools in preparation for
replacement.

Change-Id: Id5817f8265996862a7e0810b9fb9e3d78be5d066
2019-07-27 02:07:13 +00:00
Jeremy Stanley
55f657c68d Swap gitea02 into service and bring down gitea03
Add the gitea02.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 02 to 03, and remove 03 from the Ansible inventory
and comment it out in the haproxy pools in preparation for
replacement.

Change-Id: I4b51291311064c60d4bb2d90bec6e5cb90a54f3c
2019-07-26 18:00:52 +00:00
Clark Boylan
c23ac25264 Remove gitea02 from inventory so we can replace it
The global inventory is used when launching nodes so if we want to
replace a server we have to remove it from the inventory first. This is
that step for replacing gitea02.

Note that when adding it back for the new server there are some edits to
make to the playbooks as noted in the gitea sysadmin docs.

We do also remove this instance from haproxy as well to prevent unwanted
connections while we flip things over.

Change-Id: I53a3f517d46d046cb59e3185ca19ba3df55d8466
2019-07-24 20:12:16 -07:00
Jeremy Stanley
866b52f9fb Readd gitea01 to haproxy pools
Now that the replacement gitea01 server has up to date content, add
it back to the haproxy configuration.

Change-Id: I24b4659603efa1861fed1238b8eda6c3f6c11a14
2019-07-24 21:08:00 +00:00
Jeremy Stanley
5587c299ea Re-add gitea01 replacement to inventory
Add new IP addresses to inventory for the rebuild, but don't
reactivate it in the haproxy pools yet.

Note this switches the gitea testing to use a host called gitea99 so
that it doesn't conflict with our changes of the production hosts.

Change-Id: I9779e16cca423bcf514dd3a8d9f14e91d43f1ca3
2019-07-23 16:17:41 -07:00
Clark Boylan
ffcd1791bf Cleanup nodepool builder clouds.yaml
We ended up running into a problem with nodepool built control plane
images (has to do with boot from volume not allowing us to delete images
that are in use by a nova instance). We have decided to clean this up
and go back to not doing this until we can do it more properly.

Note this isn't a revert because having a group for access to control
plane clouds does seem like a good idea in general and I believe there
have been changes we'd have to resolve in the clouds.yaml files anyway.

Depends-On: https://review.opendev.org/#/c/665012/
Change-Id: I5e72928ec2dec37afa9c8567eff30eb6e9c04f1d
2019-07-22 13:55:29 -07:00
Clark Boylan
a2af942fa3 Remove gitea01 from inventory so we can replace it
The global inventory is used when launching nodes so if we want to
replace a server we have to remove it from the inventory first. This is
that step for replacing gitea01.

Note that when adding it back for the new server there are some edits to
make to the playbooks as noted in the gitea sysadmin docs.

We do also remove this instance from haproxy as well to prevent unwanted
connections while we flip things over.

Change-Id: If32405b1302353f1f262a30b7392533f86fec1e4
2019-07-22 09:20:17 -07:00
Zuul
36344bfcdd Merge "Translate gitea project creation to python" 2019-07-16 19:31:11 +00:00
Zuul
0f78ac2dcc Merge "Add proxy for registry.access.redhat" 2019-07-11 20:39:34 +00:00
Monty Taylor
caebf387b4 Translate gitea project creation to python
Sadly, as readable as the use of the uri module to do the interactions
with gitea is, more reent ansible changed how subprocesses are forked
and this makes iterating over all the projects in projects.yaml take
an incredibly long amount of time.

Instead of doing it in yaml, make a python module that takes the list
one time and does looping and requests calls. This should make it be
possible to run the actual gitea creation playbook in integration tests.

Change-Id: Ifff3291c1092e6df09ae339c9e7dddb5ee692685
2019-07-11 08:21:35 -04:00
Kevin Carter
525d21a332
Add proxy for registry.access.redhat
This change adds a proxy config for registry.access.redhat which should
assist us when gating using images provided by the publically available
registry.

Change-Id: Ica7477d63659610de852d305a63f3e78d0dd8c4f
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2019-07-10 07:31:08 -05:00
Clark Boylan
6baa9dca5c Put gitea06 back in the rotation
This server was replaced and has had its db restored from backup on
gitea01, repo dirs recreated via gitea admin ui function, and gerrit has
replicated all repo content to this server.

Put this back into the rotation in haproxy as well as the ansible
management of gitea git repos.

Change-Id: I424d0db0adf0787d5d46e264b6552d79b48f27ef
2019-06-26 16:36:57 -07:00
James E. Blair
2e5291f377 Get an LE cert for tarballs.opendev.org
Depends-On: https://review.opendev.org/663424
Change-Id: I4faa12b5d241144463ccf7ec59ef2d0b11479c35
2019-06-05 13:56:34 -07:00
Zuul
1fe34e00d4 Merge "Add control plane clouds to nodepool builder clouds.yaml" 2019-06-04 20:15:24 +00:00
James E. Blair
5faf89f566 Add haproxy-statsd to haproxy server
Build a container image with the haproxy-statsd script, and run that
along with the haproxy container.

Change-Id: I18be70d339df613bf9a72e115e80a6da876111e0
2019-05-24 15:40:28 -07:00
Monty Taylor
ff1b8a94c6 Add control plane clouds to nodepool builder clouds.yaml
In order to have nodepool build images and upload them to control
plane clouds, add them to the clouds.yaml on the nodepool-builder
hosts. Keep them out of the launcher configs by splitting the config
templates. So that we can keep our copies of things to a minimum,
create a group called "control-plane-clouds" and put bridge and nb0*
in it.

There are clouds mentions in here that we no longer use, a followup
patch will clean those up.

NOTE: Requires shifting the clouds config dict from
host_vars/bridge.openstack.org.yaml to group_vars/control-plane-clouds.yaml
in the secrets on bridge.

Needed-By: https://review.opendev.org/640044
Change-Id: Id1161bca8f23129202599dba299c288a6aa29212
2019-05-23 14:34:10 -05:00
Adam Coldrick
e9b2ca3774 Update key for SotK
Change-Id: Ic0ca12a5036fb9025f05c2a9c267da84af62dafc
2019-05-22 20:09:08 +01:00
Zuul
46c09946b4 Merge "Adds new key for diablo_rojo" 2019-05-21 23:01:30 +00:00
Kendall Nelson
ddc677db19 Adds new key for diablo_rojo
Change-Id: I3805ebcf613ba4459efe0bc28f6c4b0283eb12df
2019-05-22 00:01:16 +02:00