This refactors the existing windmill-tox-deploy job. We are basically
removing a lot of the logic from tox -edeploy and just using the venv
environment from tox. It is very much possible we may remove the
dependency on tox in the future.
Change-Id: I29a1b06c4e02e3c5f588135042f77ccc4a031a7d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This adds a DIB element to allow nodepool-builder and zuul-executor to
properly SSH into a VM launched by nodepool-launcher.
Change-Id: I5c21f6d5a9e5e0ca963aa78c8dcab14ce55365a8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because zuul uses multiple SSH keys, rename the existing gerrit SSH
key to allow for additional SSH keys.
Change-Id: Iefd507d79a26e77b6bb7b85774dd2d2a48706485
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Expose a place for users to start writing custom elements for
nodepool-builder.
Change-Id: I311fbf1c23d16939b91dd94f566dcd4509fc502e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Move cache directory for nodepool-builder to /opt/nodepool/cache, this
is helpful for people that want to setup an external volume.
Change-Id: I1967328c693a5554fe11c12c4b57f4d4613832a1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We no longer need to manually create the TMPDIR for nodepool-builder,
this is handled by ansible-role-nodepool.
Change-Id: If04ed957d2a5fea39968e9d98cd7bc5f6412a223
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Noticed a few things when we moved the configuration into the new
folder. Trivial refactor.
Change-Id: I4ed475a7a09dc6aab559b1cc308892541d11ca49
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Update our playbooks to use include_role in all places.
Change-Id: Id0dbf8ea2ef8f263c00c834f80838f05308ad702
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This is the first commit which will start to move our end user
configuration into its own git repo. For now, we can just move this
into the top-level config directory, but eventually this will be
placed in an external repo which users will then be able to clone and
modify locally.
Change-Id: I2db520b5b5349f30bd2495a7f1b686dc4de21c5f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We can use the changed_when: false logic to help make our playbook
more idempotent.
Change-Id: I502f8457b7d0756b85d9a95ff45557953ae850d9
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
It seems there is a bug in logrotate where using the -d flag doesn't
work as expected. Remove this for now, as it is blocking production
installs. But see how we can better validate logrotate is setup
properly.
Change-Id: I66959d9a6de131683a48af68ba58c08020f30e1b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
To help simplify our install process, move zuul services into a single
venv. It would be odd for a user to run 2 different versions of
service for zuul. Maybe possible, but lets not default to that.
Change-Id: Id067a4cbb57815d4d5e316a922f09805f3ff2846
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
To help simplify our install process, move nodepool services into a
single venv. This helps avoid the issue where nodepool-builder and
nodepool-launcher ansible groups could run different versions.
Change-Id: Ieda5c88069af6bfbe02812b68abf097ae65b8cea
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The zuul-executor used bubblewrap for isolation of ansible-playbook.
Add it into windmill for now, but we really should make this an
ansible-role.
Change-Id: I9a459b5a073052e85fcd80bebd142db1bc217593
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Zuul and nodepool ansible roles have removed this logic, we should
too. The service files will automagically setup the patch to our
virtualenvs.
Change-Id: I75453b72e2e4c9cb3d184f707eec7c2a337430aa
Depends-On: https://review.openstack.org/558966
Depends-On: https://review.openstack.org/558968
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The geard process is also able to use statsd, make sure we enable it.
Change-Id: Ib3694433e020f1094d876ea3dc5f240b6f14e021
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This starts building out support for statsd, which will be helpful for
users who want to start producing reports.
Currently both nodepool / zuul only support ipv4 for statsd. As a
results, limit enabling it on ipv4 hosts for now.
Change-Id: I029376330677e035a292bf95d9e7e63b57947985
Depends-On: https://review.openstack.org/554695
Depends-On: https://review.openstack.org/556209
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We no longer validate builds by using the tarball produced by
diskimage-builder. While smaller, this is more to clean up unsured
bits.
Change-Id: I3c44567f2e14382516051ebf4748559da4e0c4ac
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We no longer use this playbook, so lets delete it.
Change-Id: I271c57a1250ef430582df159c28b3020ae57e4e3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because we transmit secrets over gearman, it is best to setup SSL to
help protect that info.
Change-Id: I0dc62d6434db73c29f2581ea217db7f2cd1b191e
Depends-On: https://review.openstack.org/557121
Depends-On: https://review.openstack.org/558647
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When a user tries windmill, they have a multiple options to actually
run ansible. However, we can atleast provide a basic way to bootstrap
a host and validate our playbooks and roles are in working order.
This could also be a simple way to try newer version of ansible, under
zuul gains the ability to support that.
Change-Id: I4e97b0f10437adf7a153148cc7fa5ae45e81d4de
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Given that we are just installing CI tools, we can leave this step up
to the operator to manage them setup.
Change-Id: I965189e0b559b8a4a79663105eaf48e83cc820f1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Don't recursive copy subdirectories when collecting logs, as we don't
know what is inside them. This means we need to be more speciic of
which directories we want to copy.
Change-Id: I50691a2536d1f4127eddd133f6d99157085ed76c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We can remove the dependency on zuul-cloner to manage our roles.
Change-Id: I50f5013eacc6b6b71bb1852eebc40c5ae5807b07
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Reorganize our inventory file into a top-level directory.
Change-Id: Ice7a72cb1e5dc04053e44af9a374639587c8f5c2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This is no longer correct, and in need of replacement. Remove for now
until we get our documentation in order.
Change-Id: I90f12df4b56578ac25ab6391b0f362cb76ba7b6a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Seems we have YAML parse errors before we don't use a string for IP
addresses.
Change-Id: Ic305e52efdc88cc3674874f6431dcf8b8bf67ed3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Wrap zookeeper using the ipwrap filter, as the python kazoo client
requires [ ] brackets around ipv6 address.
This also adds the missing sections to zuul.conf
Change-Id: Ief8677a477443f0d8d68fecbf428634088f7406e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This is no longer needed and now caused our jobs to fail from
zuul-executor.
Change-Id: I51916f981a2e3584d87991a8b1d1a8597d37c915
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Zuul source code has been updated to remove symlinks outside of it
project directory.
This reverts commit 2645838f34.
Change-Id: I637e94eae759231af8323e95d004048546ac404b
Depends-On: https://review.openstack.org/553316
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Add some basic validation that we can logrotate our nodepool / zuul
services.
Change-Id: I00142be48f823d829c67f4b93f27acbe6b17d2a6
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We didn't actually include a the role in any of our zuul tasks, so we
didn't actually properly setup logrotate for zuul.
Change-Id: I698c1dc5a28b7bb5376c037f520141ef9ba3abf1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
When we moved these values into zuul jobs, it broke running windmill
locally. Set some sane defaults if they are missing from the
inventory file.
Change-Id: I313d5b62c2d8f8a3179c9dbc9dec08b391124380
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
If people are running windmill locally, we don't want to use zuul-test
for the username of our zuul services. We can add these variables to
our base job to only be used in the gate.
Change-Id: Ib6080e3ead3264d5a3d902ee9a6349dc62bb5c17
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Like we do with nodepool, add logrotate support for zuul.
Change-Id: I2ac1d5ba56d760d4b931b970610ae9f069f3a131
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Seems we missed a few hosts when adding zuul-fingergw / zuul-web,
update our inventory file.
Change-Id: I6d6f560c81efa1528e8ae23e0a94e2e0da8a0617
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Zuul contains symlinks to /opt, for testing reasons however this now
break zuul-executor and synchronize due to recent security fixes. For
now, stop pushing openstack-infra/zuul to nodes until we can get a
fix.
Change-Id: I2a707229c0d17dcbd908c2c32a09a004996d5fce
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
It is helpful to have journald logs when something goes wrong with
systemd.
Change-Id: I48765caf6917c2167da6052bcf90070b113d1f99
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
And support Depends-On for zuul/nodepool/shade/diskimage-builder/gear.
Change-Id: I48868555e0358bc13b9ece56d19926805eae230a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Zuulv3 created a new zuul-web process, update our playbooks to start
using it.
Change-Id: I4c4067fddeaef727f02d71f0a935556be56231b0
Depends-On: https://review.openstack.org/491253
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
With ubuntu-bionic nodes being online, we can start testing against
them.
Change-Id: I9d3bf5b8465d6dbae97c3909f301c7af8b9f5e4a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Nodepool now has per image build logs, start collecting them. We can
also stop having a custom builder-logging.conf file.
Change-Id: I0e424bea068c88cd97009fcf45293d820ca6936a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Paul Belanger