line from /etc/hosts
Ubuntu always uses 127.0.1.1 for that with some tricky sauce
around `hostname` depending on whether it contains '.' or not.
And when I mean `hostname` it's the one returned by `hostname`
command with no arguments.
ansible_hostname is always a single word so we can match on that.
I did not want to remove just any 127.0.1.1 in case someone
is using it for other purposes. :-)
Change-Id: I8bd3d42a5e3bd0f63336ed60a0af90d52b1650d6
Closes-bug: #1862739
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].
This change removes the Ansible code and associated CI jobs.
[1]: https://review.opendev.org/669214
Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
* HAProxy is now 1.8 in CentOS 8
* Support python3 in baremetal role
* Remove support for environments without python2 installed (this could
not have worked since we gather facts before this point)
Workarounds:
* Using CentOS 7 yum repo for Docker, with module_hotfixes
Change-Id: I30bd3d58f6224ad4c9575ba66c74deabe6895cc4
Partially-Implements: blueprint centos-rhel-8
If you do the following:
* Install legacy Docker (1.12.0) using kolla-ansible bootstrap-servers
with the Rocky release or earlier.
* Update to Docker CE, using kolla-ansible bootstrap-servers with the
Stein release or later
The package is upgraded, but docker is stopped. This prevents the 'Wait
for Docker to start' task from completing, since Docker will not start.
Seen on CentOS 7.6, Docker CE 19.03.4.
This was tested and working previously, perhaps something changed with
the Docker package.
This change fixes the issue by starting and enabling Docker after the
upgrade.
Change-Id: If6e9c91f3e8d0ec366eea7ca506c6d10dbf11c3a
Closes-Bug: #1852066
If docker_storage_driver is set in globals.yml, then kolla-ansible
bootstrap-servers is run, it fails like so:
The conditional check 'docker_storage_driver' failed. The error was: error
while evaluating conditional (docker_storage_driver): 'devicemapper' is
undefined
Ansible does not like evaluating strings as conditionals. This change switches
to using the length filter.
Change-Id: Ib95cbdac2a659e7a0d5f113fe48046de3a39dcda
Closes-Bug: #1849691
Introduce kolla_address filter.
Introduce put_address_in_context filter.
Add AF config to vars.
Address contexts:
- raw (default): <ADDR>
- memcache: inet6:[<ADDR>]
- url: [<ADDR>]
Other changes:
globals.yml - mention just IP in comment
prechecks/port_checks (api_intf) - kolla_address handles validation
3x interface conditional (swift configs: replication/storage)
2x interface variable definition with hostname
(haproxy listens; api intf)
1x interface variable definition with hostname with bifrost exclusion
(baremetal pre-install /etc/hosts; api intf)
neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network
basic multinode source CI job for IPv6
prechecks for rabbitmq and qdrouterd use proper NSS database now
MariaDB Galera Cluster WSREP SST mariabackup workaround
(socat and IPv6)
Ceph naming workaround in CI
TODO: probably needs documenting
RabbitMQ IPv6-only proto_dist
Ceph ms switch to IPv6 mode
Remove neutron-server ml2_type_vxlan/vxlan_group setting
as it is not used (let's avoid any confusion)
and could break setups without proper multicast routing
if it started working (also IPv4-only)
haproxy upgrade checks for slaves based on ipv6 addresses
TODO:
ovs-dpdk grabs ipv4 network address (w/ prefix len / submask)
not supported, invalid by default because neutron_external has no address
No idea whether ovs-dpdk works at all atm.
ml2 for xenapi
Xen is not supported too well.
This would require working with XenAPI facts.
rp_filter setting
This would require meddling with ip6tables (there is no sysctl param).
By default nothing is dropped.
Unlikely we really need it.
ironic dnsmasq is configured IPv4-only
dnsmasq needs DHCPv6 options and testing in vivo.
KNOWN ISSUES (beyond us):
One cannot use IPv6 address to reference the image for docker like we
currently do, see: https://github.com/moby/moby/issues/39033
(docker_registry; docker API 400 - invalid reference format)
workaround: use hostname/FQDN
RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4.
This is due to old RabbitMQ versions available in images.
IPv4 is preferred by default and may fail in the IPv6-only scenario.
This should be no problem in real life as IPv6-only is indeed IPv6-only.
Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will
no longer be relevant as we supply all the necessary config.
See: https://github.com/rabbitmq/rabbitmq-server/pull/1982
For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed
to work well). Older Ansible versions are known to miss IPv6 addresses
in interface facts. This may affect redeploys, reconfigures and
upgrades which run after VIP address is assigned.
See: https://github.com/ansible/ansible/issues/63227
Bifrost Train does not support IPv6 deployments.
See: https://storyboard.openstack.org/#!/story/2006689
Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c
Implements: blueprint ipv6-control-plane
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
cloud-init to manage /etc/hosts
1) Ubuntu includes a line in /etc/hosts that makes the local hostname and
nodename (if different) point to 127.0.1.1. This can break RabbitMQ,
which expects the hostname to resolve to the API network address.
2) The distribution might come with cloud-init installed, and manage_etc_hosts
configuration enabled. If so, it will override the file /etc/hosts from cloud-init
templates at every boot, which will break RabbitMQ.
This change fixes these issues.
Change-Id: I53261d0403b983ab419bd44e705b89f7b7a1c316
Closes-Bug: #1837699
Instead of changing Docker daemon command line let's change config
for Docker instead. In /etc/docker/daemon.json file as it should be.
Custom Docker options can be set with 'docker_custom_config' variable.
Old 'docker_custom_option' is still present but should be avoided.
Co-Authored-By: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Change-Id: I1215e04ec15b01c0b43bac8c0e81293f6724f278
According to Docker upstream release notes [1] MountFlags should be
empty.
1. https://docs.docker.com/engine/release-notes/#18091
"Important notes about this release
In Docker versions prior to 18.09, containerd was managed by the Docker
engine daemon. In Docker Engine 18.09, containerd is managed by systemd.
Since containerd is managed by systemd, any custom configuration to the
docker.service systemd configuration which changes mount settings (for
example, MountFlags=slave) breaks interactions between the Docker Engine
daemon and containerd, and you will not be able to start containers.
Run the following command to get the current value of the MountFlags
property for the docker.service:
sudo systemctl show --property=MountFlags docker.service
MountFlags=
Update your configuration if this command prints a non-empty value for
MountFlags, and restart the docker service."
Closes-bug: #1833835
Change-Id: I4f4cbb09df752d00073a606463c62f0a6ca6c067
This change makes kolla-ansible more compatible with
RHEL which does not provide epel-release package.
EPEL was required to install simplejson from rpm
which was an ansible requirement when used python
version was below 2.5 ([1]). This has been obsolete for
quite a time so it's a good idea to get rid of it.
This change includes update of docs to read more properly.
[1] https://docs.ansible.com/ansible/2.3/intro_installation.html
Change-Id: I825431d41fbceb824baff27130d64dabe4475d33
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
* Ubuntu ships with nfs-ganesha 2.6.0, which requires to do an rpcbind
udp test on startup (was fixed later)
* Add rpcbind package to be installed by kolla-ansible bootstrap when
ceph_nfs is enabled
* Update Ceph deployment docs with a note
Change-Id: Ic19264191a0ed418fa959fdc122cef543446fbe5
No need to touch the sudoers.d file each time
Creation and mode setting is handled by lineinfile itself
Change-Id: Ia36e21b04d3a08fab3c748f6298f142c1d73ee6d
Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
Stop showing the task has having made changes and silence warning about
not using the yum module (which we could use for the check, but not as
easily).
Change-Id: I9e3608b5db521930409a29981767f468ea234679
Kolla-Ansible populates /etc/hosts with overcloud hosts using their API
interface IP address. When configured correctly, this allows Nova to use
the API interface for live migration of instances between compute hosts.
The hostname used is from the `ansible_hostname` variable, which is a
short hostname generated by Ansible using the first dot as a delimiter.
However, Nova defaults to use the result of socket.gethostname() to
register nova-compute services.
In deployments where hostnames are set to FQDNs, for example when using
FreeIPA, nova-compute would try to reach the other compute node using
its FQDN (as registered in the Nova database), which was absent from
/etc/hosts. This can result in failures to live migrate instances if
DNS entries don't match.
This commit populates /etc/hosts with `ansible_nodename` (hostname as
reported by the system) in addition to `ansible_hostname`, if they are
different.
Change-Id: Id058aa1db8d60c979680e6a41f7f3e1c39f98235
Closes-Bug: #1830023
Several config file permissions are incorrect on the host. In general,
files should be 0660, and directories and executables 0770.
Change-Id: Id276ac1864f280554e98b937f2845bb424d521de
Closes-Bug: #1821579
When kolla-ansible bootstrap-servers is run, it executes one of the
following two tasks:
- name: Ensure node_config_directory directory exists for user kolla
file:
path: "{{ node_config_directory }}"
state: directory
recurse: true
owner: "{{ kolla_user }}"
group: "{{ kolla_group }}"
mode: "0755"
become: True
when: create_kolla_user | bool
- name: Ensure node_config_directory directory exists
file:
path: "{{ node_config_directory }}"
state: directory
recurse: true
mode: "0755"
become: True
when: not create_kolla_user | bool
On the first run, normally node_config_directory (/etc/kolla/) doesn't
exist, so it is created with kolla:kolla ownership and 0755 permissions.
If we then run 'kolla-ansible deploy', config files are created for
containers in this directory, e.g. /etc/kolla/nova-compute/. Permissions
for those files should be set according to 'config_owner_user' and
'config_owner_group'.
If at some point we again run kolla-ansible bootstrap-servers, it will
recursively set the ownership and permissions of all files in /etc/kolla
to kolla:kolla / 0755.
The solution is to change bootstrap-servers to not set the owner and
permissions recursively. It's also arguable that /etc/kolla should be
owned by 'config_owner_user' and 'config_owner_group', rather than
kolla:kolla, although that's a separate issue.
Change-Id: I24668914a9cedc94d5a6cb835648740ce9ce6e39
Closes-Bug: #1821599
Recently as part of adding support for Docker CE we added the following
task to the baremetal role:
- name: Update yum cache
yum:
update_cache: yes
become: True
when: ansible_os_family == 'RedHat'
This works fine on Ansible 2.5, but no longer works on Ansible
2.6, which complains that either the 'name' or 'list' argument
is mandatory for the yum module.
This change updates the cache later on, when installing packages.
Change-Id: I1a158bda52c4e362cb12d361d7f961cfc699b385
Closes-Bug: #1819173
Kolla Ansible's bootstrap-servers command provides support for
installing the Docker engine. This is currently done using the packages
at https://apt.dockerproject.org and https://yum.dockerproject.org.
These packages are outdated, with the most recent packages from May 2017
- docker-engine-17.05.
The source for up to date docker packages is
https://download.docker.com, which was introduced with the move to
Docker Community Edition (CE) and Docker Enterprise Edition (EE).
This change adds support to bootstrap-servers for Docker CE for CentOS
and Ubuntu.
It also adds a new variable, 'enable_docker_repo', which controls
whether a package repository for Docker will be enabled.
It also adds a new variable, 'docker_legacy_packages', which controls
whether the legacy packages at dockerproject.org will be used or the
newer packages at docker.com. The default value for this variable is
'false', meaning to use Docker CE.
Upgrading from docker-engine to docker-ce has been tested on CentOS 7.5
and Ubuntu 16.04, by running 'kolla-ansible bootstrap-servers' with
'docker_legacy_packages' set to 'false'. The upgrades were successful,
but result in all containers being stopped. For this reason, the
bootstrap-servers command checks running containers prior to upgrading
packages, and ensures they are running after the package upgrade is
complete.
As mentioned in the release note, care should be taken when upgrading
Docker with clustered services, which could lose quorum. To avoid this,
use --serial or --limit to apply the change in batches.
Change-Id: I6dfd375c868870f8646ef1a8f02c70812e8f6271
Implements: blueprint docker-ce
Even though Kolla services are configured to log output to file rather than
stdout, some stdout still occurs when for example the container re(starts).
Since the Docker logs are not constrained in size, they can fill up the
docker volumes drive and bring down the host. One example of when this is
particularly problematic is when Fluentd cannot parse a log message. The
warning output is written to the Docker log and in production we have seen
it eat 100GB of disk space in less than a day. We could configure Fluentd
not to do this, but the problem may still occur via another mechanism.
Change-Id: Ia6d3935263a5909c71750b34eb69e72e6e558b7a
Closes-Bug: #1794249
since we use chrony container to adjust time by default, we no need
to enable ntp service, this ps to disable it.
Change-Id: I2f1fd9269c9f8cfd0c98e0e903ba69de692473a0
In some cases we may want a configuration in which the kolla user's
primary group name is not the same as their username. Doing this
currently breaks the sudoers configuration, since user entries should
reference a user, or a group prefixed with a '%'.
There does not seem to be a good reason to give root privileges to the
entire group (which sometimes may be a shared group), so let's revert to
giving only the user root privileges.
See kayobe CI test [1] in which a different user and group were
configured, leading to permission denied when using kolla ansible.
[1] http://logs.openstack.org/53/581053/2/check/kayobe-overcloud-centos/a70168e
TrivialFix
Change-Id: I677778ebd0de58df0adfa2a8705f161ec5552283
In some environments it may not be desirable to modify the sudoers
configuration. This change makes this part of bootstrap-servers
optional, based on the create_kolla_user_sudoers variable.
Change-Id: I653403bfc5431741807edef57df58e05e679900b
This makes the bootstrap-servers command more idempotent, since without
the append argument set the kolla user will be removed from the docker
group before being added to it again in a later task.
TrivialFix
Change-Id: Iab0f6b5e18a103e9140631ee3ebbbb48c490bc24
In I86bf5e1df3d6568c4f1ca6f4757f08a3dd22754d, creation of the kolla user
was moved to after package installation to ensure the sudo package is
installed when required. This change does not work when python
dependencies are installed in a virtual environment however - when the
virtualenv variable is set.
This change moves the ownership change of the virtualenv to after the
kolla user has been created. It also uses the kolla_user and kolla_group
variables to set the user and group appropriately.
Change-Id: I320e5d611099ad162945a98d5505a79606da0eba
TrivialFix
By default, kolla configure docker to use an insecure connection
with the private registry. If we want to use SSL verification we need
to add an option.
Change-Id: Id1805c9cfeb499da9bb56c70028f14c6f8bb20b6
sudo package is required when we use ubuntu base on centos to deploy.
The following tasks belong to the environment check after
installation of environment-related software packages.
So, move to the post-install module.
Create kolla user
Add public key to kolla user authorized keys
Grant kolla user passwordless sudo
Ensure node_config_directory directory exists for user kolla
Ensure node_config_directory directory exists
Change-Id: I86bf5e1df3d6568c4f1ca6f4757f08a3dd22754d
Closes-Bug: #1777571
Some options can't be add in the daemon.json
The only way to configure daemon is to add option in the service file
Change-Id: I88697951ed6518f62bca30bb52288ad0e487ec03
- rename action and serial to kolla_ansible and kolla_serial
- use become instead of "sudo <command>" in shell
- Remove quota for failed_when and changed_when in rabbitmq tasks
Change-Id: I78cb60168aaa40bb6439198283546b7faf33917c
Implements: blueprint migrate-to-ansible-2-2-0
This patchset implements yamllint test to all *.yml
files.
Also fixes syntax errors to make jobs to pass.
Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799