826 Commits

Author SHA1 Message Date
Michał Nasiadka
1ccccbcf0a Bump up supported Ansible version to 2.10
Min version stays as 2.9

Change-Id: I7ec8c5eb36757248c9aa016dc7d4e495ec5bb635
2021-03-08 11:40:31 +00:00
Mark Goddard
519ca1c083 Fix installation with pip install --user
If kolla-ansible is installed via pip install --user, currently the
kolla-ansible script is unable to locate the installed playbooks.
This leads to a failure when running commands.

This change fixes the issue by checking for the user's .local directory
as a possible installation path.

This fixes some of the scenario tests which were failing after switching
to a user installation in Ifaf1948ed5d42eebaa62d7bad375bbfc12b134d5.
Most tests did not fail since the kolla-ansible script in the source
checkout was used.

Closes-Bug: #1915527

Change-Id: I5b47a146627d06bb3fe4a747c5f20290c726b0f9
2021-02-12 17:58:04 +00:00
Zuul
d6f3398538 Merge "Remove validate-install-command.sh" 2020-11-20 12:19:37 +00:00
Mark Goddard
202b4af1e9 Remove validate-install-command.sh
This file is for kolla images.

Change-Id: I2c1a21790b65cf6537380d576c0abb4d16bd369b
2020-11-20 08:42:52 +00:00
Zuul
e45cfec339 Merge "Fix kolla-ansible to work with pyenv-virtualenv" 2020-11-19 10:42:36 +00:00
zengchen228
aaab1d1b68 Fix kolla-ansible to work with pyenv-virtualenv
One of the pyenv-virtualenv-set-up aliases depends on a symlink.
It seems pyenv runs the bash script from such a path and it fails
because of a failing comparison (VIRTUAL_ENV not detected).

The VIRTUAL_ENV is ensured to be fully resolved as well for safety.

This requires readlink from GNU coreutils but all supported platforms
have it by default.

Extra comments included, as well as simplification of directory
detection - readlink handles this (not that `bin` itself was
ever a symlink...).

Closes-Bug: #1903887
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I2fe6eb13ce7be68d346b1b3b7036859f34c896c4
2020-11-18 20:41:01 +00:00
Mark Goddard
9444631078 CI: add missing --fail argument to curl
Change-Id: Ibd06726ac6edcb63a1d5d4f4148851876316dc5b
2020-11-13 17:08:44 +00:00
Mark Goddard
894f4912ac octavia: generate certificates automatically
implemented as a separate command (kolla-ansible octavia-certificates)

Implements: blueprint implement-automatic-deploy-of-octavia

Co-Authored-By: wu.chunyang <wuchunyang@yovole.com>
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>

Change-Id: I2c5b26ce9e363f35c523865904a582f7960aa682
2020-10-08 16:50:30 +02:00
zhoulinhui
06f5ae36a9 Remove kolla-cli from cleanup-host script
The kolla-cli is deprecation [1], it should be clean up from
kolla-ansible's cleanup-host script

[1]: https://review.opendev.org/#/c/749045/

Change-Id: I7072de235d9d629b0f538dc98c5258ee5f023376
2020-09-29 21:34:13 +08:00
Pierre Riteau
c5c6d995d3 Bump minimum Ansible version to 2.9
Change-Id: I5befc72a4894d625ca352b27df9d3aa84a2f5b2c
2020-09-23 17:48:01 +02:00
Zuul
b49c86848d Merge "[CI] Ensure network is set for Zun" 2020-09-14 12:18:27 +00:00
Radosław Piliszek
7a3072e94c [CI] Ensure network is set for Zun
If we don't set it, then Zun chooses one randomly (the first one
from Neutron).
This may break if it is a network that is not available on
target hosts, e.g. external via L3 agent router.

Since capsules do not support nets yet [1], this patch ensures
desired network creation order in init-runonce instead.

[1] https://bugs.launchpad.net/zun/+bug/1895263

Change-Id: Iaa113dcfb826164a2772d2c91d34ec0236be0817
2020-09-12 12:06:34 +02:00
Radosław Piliszek
b21c07ac2f [CI] Remove setup_gate.sh symlink
This is confusing as it is not meant to be used by users.
Also, various tools show duplicated matches due to both locations
containing the exact same content.

Change-Id: I2debe121f64954e57788270d3258775f29f1cbb0
2020-09-08 09:45:50 +02:00
Zuul
72fd444e31 Merge "remove obsolete configurations" 2020-08-24 11:22:32 +00:00
likui
8d458e95a4 Delete the /var/log/kolla directory should use sudo
Change-Id: Iff699b6dd9417e2e72618263641cbfa6ccb6e585
Related-Bug: #1892341
2020-08-20 18:00:31 +08:00
wu.chunyang
3c312a4d9e remove obsolete configurations
remove cluster_interface from project.
update storage_interface docs.and remove
storage_interface_address variable

Change-Id: I3f811db988234f94b5ed0cc9d24233f70784f58d
2020-08-20 00:06:49 +08:00
Zuul
9a8341c2a7 Merge "Performance: Run common role in a separate play" 2020-07-17 15:43:22 +00:00
Zuul
59b2f18de4 Merge "Support editable virtualenv installation for development" 2020-07-15 11:50:53 +00:00
Zuul
ac41906d89 Merge "CI: add prometheus-efk scenario" 2020-07-14 08:56:05 +00:00
Michal Nasiadka
bf985930d0 Evaluate PASSWORDS_FILE later
Currently seting --configdir on kolla-ansible CLI doesn't set properly the path
for the passwords file.

Change-Id: I38d215b721ec256be6cfdd6313b5ffb90c2a3f4c
Closes-Bug: #1887180
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2020-07-10 17:32:35 +02:00
Mark Goddard
f44876c406 CI: add prometheus-efk scenario
Tests prometheus, grafana, and centralised logging.

The tests could be improved in future by querying logs in elasticsearch,
and metrics in prometheus.

Change-Id: Iabad035d583d291169f23be3d71931cb260e87ae
2020-07-10 07:55:14 +00:00
Mark Goddard
56ae2db7ac Performance: Run common role in a separate play
The common role was previously added as a dependency to all other roles.
It would set a fact after running on a host to avoid running twice. This
had the nice effect that deploying any service would automatically pull
in the common services for that host. When using tags, any services with
matching tags would also run the common role. This could be both
surprising and sometimes useful.

When using Ansible at large scale, there is a penalty associated with
executing a task against a large number of hosts, even if it is skipped.
The common role introduces some overhead, just in determining that it
has already run.

This change extracts the common role into a separate play, and removes
the dependency on it from all other roles. New groups have been added
for cron, fluentd, and kolla-toolbox, similar to other services. This
changes the behaviour in the following ways:

* The common role is now run for all hosts at the beginning, rather than
  prior to their first enabled service
* Hosts must be in the necessary group for each of the common services
  in order to have that service deployed. This is mostly to avoid
  deploying on localhost or the deployment host
* If tags are specified for another service e.g. nova, the common role
  will *not* automatically run for matching hosts. The common tag must
  be specified explicitly

The last of these is probably the largest behaviour change. While it
would be possible to determine which hosts should automatically run the
common role, it would be quite complex, and would introduce some
overhead that would probably negate the benefit of splitting out the
common role.

Partially-Implements: blueprint performance-improvements

Change-Id: I6a4676bf6efeebc61383ec7a406db07c7a868b2a
2020-07-07 15:00:47 +00:00
Mark Goddard
5fb37ae584 Support editable virtualenv installation for development
An editable installation allows changes to be made to the source code
directly, and have those changes applied immediately without having to
reinstall.

    pip install -e /path/to/kolla-ansible

Change-Id: I023d96d25edd9d2fafd4415743e298af72a861a1
2020-07-02 18:15:40 +01:00
Mark Goddard
4d8d678240 Fix unexpected message without globals.d
Recently a feature was merged to support pulling in multiple
configuration files from a globals.d directory. However, if this
directory does not exist, we get the following error when executing
kolla-ansible:

    find: '/etc/kolla/globals.d': No such file or directory

This change addresses this by redirecting find command stderr to
/dev/null.

TrivialFix

Change-Id: Ie5aa511a5ebf3355817a7c3bb65b09ac5dcf2b67
2020-06-29 14:11:05 +01:00
Zuul
64f05d7a05 Merge "CI: Move NFV reqs installation to where it belongs" 2020-06-19 13:48:44 +00:00
Konstantinos Mouzakitis
f6d8c0d481 Adding support for multiple globals files
Added a spec file for this blueprint.
Changed the kolla-ansible script to accept more than one
globals.yml file. That will still be the main one but operators
will be able to create more, under the /etc/kolla/globals.d
directory.
Also added some paragraphs in the quickstart documentation
about this.
Finally, Adding a release note

Change-Id: I34eb91d0e2ed80694594b8fc6801cf8ad77da754
Implements: blueprint multiple-globals-files
2020-06-18 17:33:51 +00:00
Radosław Piliszek
7316815055 CI: Move NFV reqs installation to where it belongs
This ought to fix #1864238 for the stable branches.

Change-Id: I218905247a4a4003ecfc2c9ab3e47767bb5ab33e
Related-bug: #1864238
2020-06-12 21:33:22 +02:00
Luke Short
e8b2053f51 Do not ask for a SSH key password
This is to avoid waiting for the user to input data to continue
creating sample resources.

Change-Id: I46ff7a4779dae4c9cbe157e1712afa4e53be269e
Signed-off-by: Luke Short <ekultails@gmail.com>
2020-05-28 20:57:53 -04:00
Zuul
05e3597cfa Merge "Check that used Ansible can see Kolla Ansible" 2020-05-24 13:39:58 +00:00
xiaojueguan
593b7b4ce4 fix can not generate ovs-dpdk.conf
Change-Id: I0ecc10f8035f421d7dc1132c2b29db0d9cc84cdf
Closes-Bug: #1879125
2020-05-17 16:28:59 +08:00
Radosław Piliszek
32fc2599a6 Check that used Ansible can see Kolla Ansible
Fix-feature following up on the original check [1] to make it
test the correct interpreter.

Additionally, this change removes last, unneeded call to
random python - getting script directory is perfectly
doable in bash.

All checks are done from Python, not Ansible, due to its
performance. Python version feels snappy (0.2 s to check),
compared to sluggish Ansible (2.0 s to check).
What is more, relying on Ansible would require hacky solutions
to e.g. prevent custom config from interfering with it.
We might be willing to steer Ansible from Python in the future
anyhow.

[1] Icf0399d21b3fde8d530d73e6e7ee4a57665da276

Change-Id: Ib8f2e6b6672e7c06aa94bc226c4d72640d25d8c2
Closes-Bug: #1856346
2020-04-27 17:18:31 +02:00
Radosław Piliszek
3036c7cfda CI: Make bashate happy
Bashate warned on init-swift.sh because of E043 -- arithmetic
compound has inconsistent return semantics: (( next_port++ ))
New Zuul reported that on every change as a warning.
This is fixed here.

This change makes Bashate always produce errors so that we do not
introduce such warnings again.

Change-Id: I40166b377ec2580e17901375b636183bca492d3a
2020-04-24 15:57:55 +02:00
Zuul
5273828fae Merge "Remove support for Python 2 on the host" 2020-04-21 17:01:25 +00:00
Zuul
9f54c6fd1b Merge "Remove support for CentOS 7" 2020-04-20 16:57:53 +00:00
Mark Goddard
284f492861 Remove support for Python 2 on the host
Drops support for creating Python 2 virtualenvs in bootstrap-servers,
and looking for a python2 interpreter in the kolla-ansible script.

Also forces the use of Python 3 as the remote interpreter in CI on
Debian and Ubuntu hosts, since they typically symlink the unversioned
interpreter to python2.7.

Change-Id: Id0e977de381e7faafed738674a140ba36184727e
Partially-Implements: blueprint drop-py2-support
2020-04-20 16:18:31 +00:00
Michal Nasiadka
8a0740df97 OVN Support
Implement OVN Ansible role.

Implements: blueprint ovn-controller-neutron-ansible

Depends-On: https://review.opendev.org/713422
Change-Id: Icd425dea85d58db49c838839d8f0b864b4a89a78
2020-04-09 07:40:12 +02:00
Mark Goddard
f4e20a1f16 Remove support for CentOS 7
CentOS 8 support is now fairly complete - time to drop CentOS 7.

Partially-Implements: blueprint centos-rhel-8

Change-Id: I940b1d3eceb98e16fa366c243672f588b1412d70
2020-04-08 17:05:10 +01:00
Zuul
fa49143f10 Merge "Update hacking for Python3" 2020-04-06 06:43:46 +00:00
Andreas Jaeger
454489762c Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Remove hacking and friends from lower-constraints, they are not needed
during installation.

Change-Id: I7ef5ac8a89e94f5da97780198619b6facc86ecfe
2020-04-04 17:00:17 +02:00
Radosław Piliszek
ed1b74a35b CI: Avoid tox (and clean up gate setup)
We don't need tox nor tell our users to use it when doing kolla
builds.
Tox is going away from base infra images.
It's already gone from aarch64 (arm64) ones.

Change-Id: I2eb5203ad93c011b8806f4b6fb56db081c14a2cb
2020-04-03 18:04:52 +01:00
Doug Szumski
b758883418 Support setting Kafka storage volume
Not everyone wants Kafka data stored on a Docker volume. This
change allows a user to flexibly control where the data is stored.

Change-Id: I2ba8c7a85c7bf2564f954a43c6e6dbb3257fe902
2020-03-30 11:00:03 +02:00
Marcin Juszkiewicz
9bde29a30b cirros: upgrade to 0.5.1
We released CirrOS 0.5.1. Time to move then.

Change-Id: Ibca24836f19b3cbf6166fa39a3702883938feda8
2020-03-21 15:40:10 +00:00
Radosław Piliszek
e03f06c081 Do not enforce vxlan in init-runonce
The affected command was meant to create a tenant network, so let
us really test it this way.

Not marking CI, because someone may be using this script.

Change-Id: I5abe46948992121a11a36f941d4f8fac1caa92b1
2020-03-20 14:51:02 +01:00
Zuul
0718a1a90c Merge "CI: Enable fluentd in MariaDB jobs" 2020-03-19 17:19:06 +00:00
Zuul
270fb4cf0b Merge "CentOS 8: Use master image tag by default" 2020-03-19 12:55:34 +00:00
Michal Nasiadka
866a6ba16a CI: Enable fluentd in MariaDB jobs
Since fluentd is disabled in MariaDB jobs - haproxy logs are not getting
populated.

Change-Id: I56b3fc1be6940d97905cdb2c4452b846f106c071
Depends-on: https://review.opendev.org/713704
2020-03-19 09:01:23 +00:00
Mark Goddard
bab770a88b CentOS 8: Use master image tag by default
While supporting both CentOS 7 and 8, we used the tag 'master-centos8'
for CentOS 8 images. We are now ready to drop CentOS 7 support, and
Kolla is switching to publish CentOS 8 images using the master tag on
the master branch, so we should use this.

Depends-On: https://review.opendev.org/713265

Partially-Implements: blueprint centos-rhel-8

Change-Id: I07d2c285e3214a6dc827a8e8eacf263048ee099b
2020-03-16 15:58:55 +00:00
Zuul
5a11f14b1b Merge "CI: Debug init-runonce" 2020-03-15 20:18:18 +00:00
Radosław Piliszek
525a7f326f CI: Use network mode = host for builds
This avoids issues with IPv6 not enabled in docker.

Change-Id: I0faa1df8a04acd1746fa8a5f0d8455f819c254b5
2020-03-03 13:15:30 +01:00
Radosław Piliszek
29e4ef1374 CI: Debug init-runonce
Also adds gawk for timestamping.

This helps to correlate init events with failures elsewhere.

Change-Id: I22fdb683ecf9870b2d66fedd6b40b7004317130a
2020-03-01 09:57:35 +01:00