We get deprecation warnings from ansible about use
of python2 on xenial hosts. Rather than setting
ansible_python_interpreter to python3 on a host by
host basis, set it globally to python3.
Set it to python for the one host that's too old,
refstack.openstack.org, which is running on trusty
which only has python3.4.
Change-Id: I4965d950c13efad80d72912911bc7099e9da1659
Zuul is publishing lovely container images, so we should
go ahead and start using them.
We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.
Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.
Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
We run puppet with ansible now pretty much all the time. It's not
helpful for the puppet output to go to syslog on the remote host.
What's more helpful is for it to come back to the stdout in the
ansible playbook so that we can see it.
Also turn off ansi color from the output.
Depends-On: https://review.opendev.org/721732
Change-Id: I604081d5400bd53b8dda5a3a7685323c1443991b
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.
Add the ability to override the keys for the zuul user.
Remove openstack_project::server, it doesn't do anything.
Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.
Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
We use project-config for gerrit, gitea and nodepool config. That's
cool, because can clone that from zuul too and make sure that each
prod run we're doing runs with the contents of the patch in question.
Introduce a flag file that can be touched in /home/zuulcd that will
block zuul from running prod playbooks. By default, if the file is
there, zuul will wait for an hour before giving up.
Rename zuulcd to zuul
To better align prod and test, name the zuul user zuul.
Change-Id: I83c38c9c430218059579f3763e02d6b9f40c7b89
We want to trigger nameserver updates when we merge patches
to zone files.
The zuul zone repo is currently managed by infra-core. We need to
make an improvement to zuul before we can offload core role there
to the zuul-maint team.
Change-Id: I6192f2499465844ccf2a1f903a8897458814da5d
We removed nb01 with I18ab9834ad4da201774e0abef56f618cd7839d36 and
replaced it with nb04; open the firewall for it.
Change-Id: I7138ee8744d978388b95e35ddd767cc97a5f5a87
As part of OpenDev rename, a lot of links were changed.
A couple of URLs point to old locations, update them.
This list was done while grepping for "openstack-infra" and fixing
locations that are wrong.
Change-Id: I313d76284bb549f1b2c636ce17fa662c233c0af9
Currently we deploy the openstacksdk config into ~nodepool/.config on
the container, and then map this directory back to /etc/openstack in
the docker-compose. The config-file still hard-codes the
limestone.pem file to ~nodepool/.config.
Switch the nodepool-builder_opendev group to install to
/etc/openstack, and update the nodepool config file template to use
the configured directory for the .pem path.
Also update the testing paths.
Story: #2007407
Task: #39015
Change-Id: I9ca77927046e2b2e3cee9a642d0bc566e3871515
This is a start at ansible-deployed nodepool environments.
We rename the minimal-nodepool element to nodepool-base-legacy, and
keep running that for the old nodes.
The groups are updated so that only the .openstack.org hosts will run
puppet. Essentially they should remain unchanged.
We start a nodepool-base element that will replace the current
puppet-<openstackci|nodepool> deployment parts. For step one, this
grabs project-config and links in the elements and config file.
A testing host is added for gate testing which should trigger these
roles. This will build into a full deployment test of the builder
container.
Change-Id: If0eb9f02763535bf200062c51a8a0f8793b1e1aa
Depends-On: https://review.opendev.org/#/c/710700/
I forgot this in some of the prior changes that moved afsmon and
afs-release.py to this host, and those jobs send stats.
Change-Id: Ifacf69e7fef5b54a03d43272e9cc01b6fbe8e845
We have a single vhost for zuul-ci.org and zuulci.org, so we should
request a cert with all 4 hostnames.
We also have a separate vhost to handle the git.zuul-ci.org redirect;
add a cert request for that so we can manage it with LE.
Change-Id: Ia2ba3d3ad4f5ab0356ede371d94af3c77a89eda1
I was trying to simplify things by having a restricted shell script
run by root. However, our base-setup called my bluff as we also need
to setup sshd to allow remote root logins from specific addresses.
It's looking easier to create a new user, and give it sudo permissions
to run the vos release script.
Change-Id: If70b27cb974eb8c1bafec2b7ef86d4f5cba3c4c5
I wasn't correctly sourcing the key; it has to come from hostvars as
it is in a different play on different hosts. This fixes it.
We also need to not have the base roles overwrite the authorized_keys
file each time. The key we provision can only run a limited script
that wraps "vos release".
Unfortunately our gitops falls down a bit here because we don't have
full testing for the AFS servers; put this on the todo list :) I have
run this manually for testing.
Change-Id: I0995434bde7e43082c01daa331c4b8b268d9b4bc
This change adds a proxy config for quay which should assist
us when gating using images provided by the publically
available registry.
Change-Id: I971705e59724e70bd9d42a6920cf4f883556f673
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This can be used in an apache vhost later, but should be fine to
merge now.
Depends-On: https://review.opendev.org/673902
Change-Id: Ic2cb7585433351ec1bdabd88915fa1ca07da44e7
Add the gitea07.opendev.org and gitea08.opendev.org servers into the
haproxy pools now that they've been seeded with current data. Remove
the create repos task disable list entries for them as well.
Change-Id: I69390e6a32b01cc1713839f326fa930c376282af
Add the gitea05.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 05 to 07 and 08, and remove 07 and 08 from the
Ansible inventory and comment them out in the haproxy pools in
preparation for replacement.
To the casual observer it may appear gitea06 is being skipped, but
it was replaced first out of sequence due to filesystem corruption
during the PTG. The increased performance of the 75% of the nodes
which have already been replaced means we can get by doing the final
25% at the same time (so two servers at once).
Change-Id: Ia49157c16582b7ed0dbef3eb9d07bf7f1d4450b9
Add the gitea04.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 04 to 05, and remove 05 from the Ansible inventory
and comment it out in the haproxy pools in preparation for
replacement.
Change-Id: I4cd1fef399e527771a26efee8a39952694f3ce6b
Add the gitea03.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 03 to 04, and remove 04 from the Ansible inventory
and comment it out in the haproxy pools in preparation for
replacement.
Change-Id: Id5817f8265996862a7e0810b9fb9e3d78be5d066
Add the gitea02.opendev.org server into the haproxy pools now that
it's been seeded with current data. Switch the create repos task
disable list from 02 to 03, and remove 03 from the Ansible inventory
and comment it out in the haproxy pools in preparation for
replacement.
Change-Id: I4b51291311064c60d4bb2d90bec6e5cb90a54f3c
The global inventory is used when launching nodes so if we want to
replace a server we have to remove it from the inventory first. This is
that step for replacing gitea02.
Note that when adding it back for the new server there are some edits to
make to the playbooks as noted in the gitea sysadmin docs.
We do also remove this instance from haproxy as well to prevent unwanted
connections while we flip things over.
Change-Id: I53a3f517d46d046cb59e3185ca19ba3df55d8466
Now that the replacement gitea01 server has up to date content, add
it back to the haproxy configuration.
Change-Id: I24b4659603efa1861fed1238b8eda6c3f6c11a14
Add new IP addresses to inventory for the rebuild, but don't
reactivate it in the haproxy pools yet.
Note this switches the gitea testing to use a host called gitea99 so
that it doesn't conflict with our changes of the production hosts.
Change-Id: I9779e16cca423bcf514dd3a8d9f14e91d43f1ca3
We ended up running into a problem with nodepool built control plane
images (has to do with boot from volume not allowing us to delete images
that are in use by a nova instance). We have decided to clean this up
and go back to not doing this until we can do it more properly.
Note this isn't a revert because having a group for access to control
plane clouds does seem like a good idea in general and I believe there
have been changes we'd have to resolve in the clouds.yaml files anyway.
Depends-On: https://review.opendev.org/#/c/665012/
Change-Id: I5e72928ec2dec37afa9c8567eff30eb6e9c04f1d
The global inventory is used when launching nodes so if we want to
replace a server we have to remove it from the inventory first. This is
that step for replacing gitea01.
Note that when adding it back for the new server there are some edits to
make to the playbooks as noted in the gitea sysadmin docs.
We do also remove this instance from haproxy as well to prevent unwanted
connections while we flip things over.
Change-Id: If32405b1302353f1f262a30b7392533f86fec1e4
Sadly, as readable as the use of the uri module to do the interactions
with gitea is, more reent ansible changed how subprocesses are forked
and this makes iterating over all the projects in projects.yaml take
an incredibly long amount of time.
Instead of doing it in yaml, make a python module that takes the list
one time and does looping and requests calls. This should make it be
possible to run the actual gitea creation playbook in integration tests.
Change-Id: Ifff3291c1092e6df09ae339c9e7dddb5ee692685
This change adds a proxy config for registry.access.redhat which should
assist us when gating using images provided by the publically available
registry.
Change-Id: Ica7477d63659610de852d305a63f3e78d0dd8c4f
Signed-off-by: Kevin Carter <kecarter@redhat.com>
This server was replaced and has had its db restored from backup on
gitea01, repo dirs recreated via gitea admin ui function, and gerrit has
replicated all repo content to this server.
Put this back into the rotation in haproxy as well as the ansible
management of gitea git repos.
Change-Id: I424d0db0adf0787d5d46e264b6552d79b48f27ef
Build a container image with the haproxy-statsd script, and run that
along with the haproxy container.
Change-Id: I18be70d339df613bf9a72e115e80a6da876111e0
In order to have nodepool build images and upload them to control
plane clouds, add them to the clouds.yaml on the nodepool-builder
hosts. Keep them out of the launcher configs by splitting the config
templates. So that we can keep our copies of things to a minimum,
create a group called "control-plane-clouds" and put bridge and nb0*
in it.
There are clouds mentions in here that we no longer use, a followup
patch will clean those up.
NOTE: Requires shifting the clouds config dict from
host_vars/bridge.openstack.org.yaml to group_vars/control-plane-clouds.yaml
in the secrets on bridge.
Needed-By: https://review.opendev.org/640044
Change-Id: Id1161bca8f23129202599dba299c288a6aa29212