193 Commits

Author SHA1 Message Date
Michal Nasiadka
2128075c6e Ansible lint related fixes
Change-Id: I146ea3d84efb83ec5d7405644ad372e57ecafc1e
2020-05-12 17:39:07 +00:00
Dincer Celik
4b5df0d866 Introduce /etc/timezone to Debian/Ubuntu containers
Some services look for /etc/timezone on Debian/Ubuntu, so we should
introduce it to the containers.

In addition, added prechecks for /etc/localtime and /etc/timezone.

Closes-Bug: #1821592
Change-Id: I9fef14643d1bcc7eee9547eb87fa1fb436d8a6b3
2020-04-09 18:53:36 +00:00
Mark Goddard
96151a35d0 Host OS prechecks follow up
We only log the release in the 'Checking host OS release or version'
precheck, but we allow either the release or version to be included in
the list. For example, on CentOS 7:

    CentOS release Core is not supported. Supported releases are: 8

Include the version in the failure message too.

Change-Id: I0302cd4fc94a0c3a6aa1dbac7b9fedf37c11b81e
Related: blueprint improve-prechecks
2020-03-11 17:08:17 +00:00
Mark Goddard
d20c65ed48 Check supported host OS distributions in prechecks
This should help to ensure that users are running tested and supported
host OS distributions.

Change-Id: I6ee76463d284ad4f3646af1c7ec2b7e50e2f3b15
Partially-Implements: blueprint improve-prechecks
2020-03-10 11:04:29 +00:00
Michal Nasiadka
4e6fe7a6da Remove kolla-ceph
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].

This change removes the Ansible code and associated CI jobs.

[1]: https://review.opendev.org/669214

Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
2020-02-11 11:42:06 +01:00
Zuul
39c09d0b60 Merge "Adds maximum supported version check for Ansible" 2020-01-22 13:36:34 +00:00
Mark Goddard
5fb10e08fe Ansible lint: use command module instead of shell
Change-Id: Ibf40216b847f103e383f19fe1ef608a75fcfd452
Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
2020-01-13 10:45:10 +00:00
Mark Goddard
9e3c774371 Ansible lint: remove comparison with true or false
Change-Id: I3caa4581ba276082e859f18aaa6638472f5fbe49
Co-Authored-By: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
2020-01-13 10:33:43 +00:00
Dincer Celik
63ab53195c Adds maximum supported version check for Ansible
Maximum supported version is set to 2.9

Updated the minimum supported version to 2.8

Implements: blueprint ansible-max-version

Change-Id: I97cc95e37f49886e6d74f2d5a789b923b14b5a2d
2020-01-09 09:54:22 +03:00
Michal Nasiadka
1009931162 Change local_action to delegate_to: localhost
As part of the effort to implement Ansible code linting in CI
(using ansible-lint) - we need to implement recommendations from
ansible-lint output [1].

One of them is to stop using local_action in favor of delegate_to -
to increase readability and and match the style of typical ansible
tasks.

[1]: https://review.opendev.org/694779/

Partially implements: blueprint ansible-lint

Change-Id: I46c259ddad5a6aaf9c7301e6c44cd8a1d5c457d3
2019-11-22 15:04:44 +00:00
Radosław Piliszek
bc053c09c1 Implement IPv6 support in the control plane
Introduce kolla_address filter.
Introduce put_address_in_context filter.

Add AF config to vars.

Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]

Other changes:

globals.yml - mention just IP in comment

prechecks/port_checks (api_intf) - kolla_address handles validation

3x interface conditional (swift configs: replication/storage)

2x interface variable definition with hostname
(haproxy listens; api intf)

1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)

neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network

basic multinode source CI job for IPv6

prechecks for rabbitmq and qdrouterd use proper NSS database now

MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)

Ceph naming workaround in CI
TODO: probably needs documenting

RabbitMQ IPv6-only proto_dist

Ceph ms switch to IPv6 mode

Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)

haproxy upgrade checks for slaves based on ipv6 addresses

TODO:

ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.

ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.

rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.

ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.

KNOWN ISSUES (beyond us):

One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN

RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982

For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227

Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689

Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-10-16 10:24:35 +02:00
Zuul
ff86c2f2e3 Merge "Implement TLS encryption for internal endpoints" 2019-09-12 09:20:54 +00:00
Mark Goddard
8c8adb0e45 Update minimum version of Ansible to 2.6
This is required for the dict2items filter.

Change-Id: I60a04e839bf06506ff36c2631a286130d5fde972
2019-09-11 17:09:32 +01:00
Krzysztof Klimonda
b0ecd8b67c Implement TLS encryption for internal endpoints
This review is the first one in a series of patches and it introduces an
optional encryption for internal openstack endpoints, implementing part
of the add-ssl-internal-network spec.

Change-Id: I6589751626486279bf24725f22e71da8cd7f0a43
2019-08-22 16:39:21 -07:00
Mark Goddard
0a769dc30b Bump minimum Ansible version to 2.5
This is necessary for some Ansible tests which were renamed in 2.5 -
including 'version' and 'successful'.

Change-Id: Iacf88ef5589c7571fcf56ba8b99d3dbe76975195
2019-07-01 09:38:01 +01:00
Mark Goddard
b123bf6621 Use become for all docker tasks
Many tasks that use Docker have become specified already, but
not all. This change ensures all tasks that use the following
modules have become:

* kolla_docker
* kolla_ceph_keyring
* kolla_toolbox
* kolla_container_facts

It also adds become for 'command' tasks that use docker CLI.

Change-Id: I4a5ebcedaccb9261dbc958ec67e8077d7980e496
2019-06-06 19:04:58 +01:00
Marcin Juszkiewicz
1c866201e3 prechecks: be nice, use print in Py3 way
Change-Id: I7a99567b3d16830e1db641e82a7c9e4eca1ae4e6
2019-05-20 12:53:13 +02:00
Raimund Hook
84ea42bd7c Updating Jinja filters to conform to Ansible 2.5+
Since Ansible 2.5, the use of jinja tests as filters has been
deprecated.

I've run the script provided by the ansible team to 'fix' the
jinja filters to conform to the newer syntax.

This fixes the deprecation warnings.

Change-Id: I844ecb7bec94e561afb09580f58b1bf83a6d00bd
Closes-bug: #1827370
2019-05-02 14:58:09 +01:00
caoyuan
9223deeecd Use correct variable for default certificate paths
The variable {{ node_config_directory }} is used for the configuration
directory on the remote hosts, and should not be used for paths on the
deploy host (localhost).

This changes the default value of the TLS certificate and CA file to
reference {{ CONFIG_DIR }}, in line with the directory used for
admin-openrc.sh (as of I0709482ead4b7a67e82796e17f85bde151e71bc0).

This change also introduces a variable, {{ node_config }}, that
references {{ CONFIG_DIR | default('/etc/kolla') }}, to remove
duplication.

Change-Id: Ibd82ac78630ebfff5824c329d7399e1e900c0ee0
Closes-Bug: #1804025
2018-11-19 16:25:28 +00:00
Paul Bourke
a080bb1006 Allow the deployment group to be optional
This small change prevents the deployment group from being mandatory.

Change-Id: I5693fbad0d5677ae341121619b6ec2e8738cffb7
2018-10-25 16:10:26 +01:00
Michal Nasiadka
6fad16ebb0 Stop reporting changed in prechecks
Change-Id: I12f7244aa91ccd7e287320e7edb09e0d04c01017
2018-08-02 12:34:50 +02:00
caoyuan
0dd487fdb1 Bump ansible_version_min to 2.4
the include_tasks is the new feature in version 2.4[0], and
kolla use it[1], the ansible_version_min should be updated
to version 2.4

[0]: https://docs.ansible.com/ansible/2.4/include_tasks_module.html
[1]: https://review.openstack.org/#/c/584728/

Change-Id: I273ac9e8295bfe3fb97af620c1e48e9fbbf06c6e
2018-07-30 08:38:39 +08:00
Jeffrey Zhang
b51eeed89e Use include_tasks instead of include
include is marked as deprecated since ansible 2.4[0]

[0] https://docs.ansible.com/ansible/2.4/include_module.html#deprecated

Co-Authored-By: confi-surya <singh.surya64mnnit@gmail.com>
Change-Id: Ic9d71e1865d1c728890625aeddf424a5734c0a8a
2018-07-25 23:57:22 +08:00
Jeffrey Zhang
c567055176 Fix ansible warning
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks

Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
2018-05-11 02:54:02 +00:00
Mark Giles
a4906506db Verify external VIP set when TLS enabled.
The documentation states that kolla_internal_vip_address and
kolla_external_vip_address must be different when TLS is enabled.
If this is not true, the deploy will be successful but will not
function correctly.  This change adds a precheck to catch this
configuration problem prior to deployment.

Change-Id: I6c9e5fd5f434f7225bd0390aff116fa12da96934
2018-03-28 11:29:41 -04:00
Eduardo Gonzalez
ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Zuul
96ede1eb55 Merge "Don't check api_interface for 'deployment' group" 2018-02-18 08:41:34 +00:00
Zuul
e720e10fee Merge "Fix the Prechecks failed while checking docker version" 2018-02-14 14:13:38 +00:00
Vladislav Belogrudov
722185a424 Don't check api_interface for 'deployment' group
'deployment' hosts don't need api interface. Previous fix checked
if hosts belong global groups like [compute] or [control] which
is not correct because uses can create own groups and the check
won't work there at all. Here the logic is reversed - we blacklist
groups that don't need api_interface.

Change-Id: Ibaa94004016ba41a2ae5700c48522e254c6ee239
Closes-Bug: #1732846
2018-02-14 11:53:52 +00:00
Zuul
49360f0c35 Merge "Add a precheck to ensure nscd is not running" 2018-01-28 07:30:00 +00:00
Alexandru Bogdan Pica
8e3b79440c Implement external MariaDB and pre-configured Databases support
This change allows the following use cases:

1. Using an already-configured MariaDB / MySQL server / Cluster
2. Using already-created DB users, without requiring root DB access.

Update: added external mariadb precheck

Change-Id: I78b0d178306d7c5293b0bf53e445f19f18b4b824
Implements: blueprint external-mariadb-support.
Closes-Bug: #1603121
2018-01-23 13:07:40 +00:00
chenxing
d1ddeddb67 Fix the Prechecks failed while checking docker version
In Docker version 18.01.0-ce, "docker version" command uses the
"\t" to format the result, which will cause the prechecks failed.
There is a workaround that using "docker --version" command
instead of "docker version" can get everything running.
Also, using "docker --version" to check the version of docker is
more specific and effective.

backport: pike ocata

Change-Id: Ie5efe1888afc3be11441a88d627db62e1575f3ed
Closes-Bug: #1742869
2018-01-22 16:58:48 +08:00
Paul Bourke
8a2048906d Add a precheck to ensure nscd is not running
The 'name service caching daemon (nscd)' seems to effectively disable
Docker's user namespace when '/run' is bind mounted. This can cause
issues with Kolla containers that use this mount, if the uid the
container runs as exists on the system, they clash and container will
either fail to start, have sudo issues, etc.

Change-Id: I9ac6bdb193f1520d025251d3abfa761700481cc9
Closes-Bug: #1723934
2018-01-15 17:47:27 +00:00
Mark Goddard
be4568334c Fix failure handling in prechecks
In a HAProxy precheck task, the command module is used, and the results
inspected by checking the stdout attribute of the result. However, if
the command fails (non-zero exit code), in some cases there may not be a
stdout attribute in the result object. This causes an AttributeError and
prevents ansible's useful diagnostic output from being displayed.

Change-Id: Id502b5d0b71fe2150a29df43154c925dca96ef06
Trivial Fix
2017-12-14 14:05:39 +00:00
Zhijiang Hu
1618962adf Do not check api_interface for deployment group
Close-Bug: 1732846

Change-Id: I6489599453ae56a60281ecbae9ce85aa59b1162a
2017-11-23 03:08:35 -05:00
Duong Ha-Quang
26b2c2d9e9 Specify 'become' to necessary tasks (general roles)
Add config_owner_user and config_owner_group to group_vars/all,
which is user and group of Kolla configuration files in /etc/kolla.

Add become to post-deploy playbook.

Add become to only neccesary tasks in roles:
- certificate
- common
- destroy
- haproxy
- mariadb
- memcached
- rabbitmq

Change-Id: I2aba745a6e3928c52642f64551470fd08cbfd058
Partial-Implements: blueprint ansible-specific-task-become
2017-09-25 06:52:04 +00:00
Jenkins
641a7f2790 Merge "Fix prechecks fail with docker not in deployment host" 2017-09-20 16:33:43 +00:00
Hongbin Lu
3daeea37ea Upgrade from docker-py to docker
The pypi package 'docker-py' [1] has been renamed to 'docker' [2].
It is better to move to the new 'docker' package because the old
package will be deprecated and all the new features will go into
the new package only.

Package 'docker' has been added to requirements [3]. The old
package 'docker-py' is still allowed to be in the global requirements
during the transition period but it should be removed after all or
most of the projects finsih the migration.

[1] https://pypi.python.org/pypi/docker-py
[2] https://pypi.python.org/pypi/docker
[3] https://review.openstack.org/#/c/423715/

Change-Id: Ibcd5a57a1fbf55dcc5a690e41f20917f95b63da0
2017-07-10 14:19:28 +00:00
Bertrand Lallau
a9113fc466 Avoid launching "local" actions too many times
In order to speed up deployment time some "local" actions should be run
only once using 'run_once: True'.
This will decrease deployment time in case of multihost configuration.

Change-Id: I6015d772d35c15e96c52f577013b6e41197cb41a
2017-07-05 15:55:39 +02:00
Eduardo Gonzalez
3b7ab63a19 Fix prechecks fail with docker not in deployment host
This changes only evaluates baremetal host group
to look for docker versions.

Change-Id: I64a7314b332958d003f88a00aa73ba79a760a051
Closes-Bug: #1700986
2017-06-29 09:18:33 +01:00
Jeffrey Zhang
00a42b8cbe Bump min Ansible version to 2.2.0
Different Y stream version in Ansible is not compatible. Since ansible
2.2.0 is released for a while. It is time for kolla-ansible to bump the
min Ansible version. Then we can remove the annoying WARNING message
in deploying logs and use the new features.

Change-Id: I99f3c5678f6d2d1f93d61c660fbd166184ff6422
2017-06-14 06:57:04 +08:00
Jenkins
31cc7926e7 Merge "Update docker_py_version_min version" 2017-06-04 00:45:47 +00:00
Jenkins
841bc75ae2 Merge "Revert "Add jinja2-py version check in pre_checks"" 2017-05-30 17:39:17 +00:00
Eduardo Gonzalez
15a5d2cce9 Revert "Add jinja2-py version check in pre_checks"
This reverts commit 64999de5b93e393518771f86ddc8673cc7dd4e55.

Change-Id: I7d706708d476101bf9cfb43cc4cde9e41e716986
2017-05-30 15:41:16 +00:00
Jenkins
c8da11c242 Merge "Add jinja2-py version check in pre_checks" 2017-05-29 07:53:08 +00:00
Bertrand Lallau
afdd11b9a2 Generalize api_interface_address variable usage
Useful api_interface_address variable has been define here:
https://github.com/openstack/kolla-ansible/blob/master/ansible/group_vars/all.yml#L57
In order to simplify codebase we must use it as much as possible.

Change-Id: I18fec19bf69e05a22a4142a9cd1165eccd022455
2017-05-23 08:35:15 +00:00
shaofeng_cheng
5f197ed496 Update docker_py_version_min version
Update docker_py_version_min version to 1.8.1.

see:
https://github.com/openstack/kolla-ansible/blob/master/requirements.txt

Change-Id: I0780de04826d50e1bfe749f66ab8fb6dae2a359f
2017-05-04 17:14:36 +08:00
shaofeng_cheng
64999de5b9 Add jinja2-py version check in pre_checks
Jinja2-py version is missing in pre_checks.

Change-Id: I047722be3d7b4819eade6c675d79a898a2f0024b
2017-04-07 16:04:33 +08:00
Jenkins
9d0daa015a Merge "Enable sanity checks from kolla-ansible" 2017-03-09 17:31:51 +00:00
Paul Bourke
5418ada148 Enable sanity checks from kolla-ansible
Add a new subcommand 'check' to kolla-ansible, used to run the
smoke/sanity checks.

Add stub files to all services that don't currently have checks.

Change-Id: I9f661c5fc51fd5b9b266f23f6c524884613dee48
Partially-implements: blueprint sanity-check-container
2017-03-09 10:37:06 +00:00