144 Commits

Author SHA1 Message Date
Mark Goddard
085484478a Remove stop-chrony.yml
Remove transitional support for stopping the chrony container. This was
originally added for https://storyboard.openstack.org/#!/story/2005272.

Change-Id: Ief43e723b8f4b97fdcd5960aabfb7208beb0b7cd
Story: 2006949
Task: 37630
2019-11-26 13:44:09 +00:00
Mark Goddard
5977a44119 Fix baremetal serial console autoenable
If ironic_serial_console_autoenable is set to true, ironic compute node
serial consoles are automatically enabled during kayobe overcloud post
configure. However, the baremetal-compute-serial-console.yml playbook
expects a variable called cmd to be defined, and set to 'enable' in
order to enable serial consoles. This patch fixes the issue.

We also add a conditional to only import the playbook when ironic is
enabled in the overcloud.

Change-Id: Ice26a113f9023b1a25a4af0552d07c5e95e64556
Story: 2006662
Task: 36924
2019-10-21 14:16:20 +01:00
Zuul
dea732e7d5 Merge "Fix container image build with multiple regexes" 2019-10-02 15:26:47 +00:00
Mark Goddard
1e5fb00754 Fix container image build with multiple regexes
We don't need to provide additional quotes for passing these arguments
to playbooks, since this is now done automatically via
kayobe.utils.quote_and_escape.

Change-Id: I9588b58449e8c8f8eab85ce01b4e130adc150ac6
Story: 2006475
Task: 36416
2019-09-11 18:34:15 +01:00
Mark Goddard
45ef33ba91 Use earliest-version 6.0.0 for Stein reno
Change-Id: I7d6bbc700136a332e4e93d3c8e347f7efad196fd
2019-08-30 15:11:24 +01:00
Zuul
f005dc2162 Merge "Set provisioning and cleaning networks as non-shared" 2019-08-28 19:24:58 +00:00
Mark Goddard
f91cf43876 Set provisioning and cleaning networks as non-shared
Modifies provisioning and cleaning networks in multi-tenant ironic
environments to be non-shared. Flat networks remain shared.

To apply the change to an existing environment, run 'kayobe overcloud
post configure'.

Change-Id: Ie59f0c729381e5bea112ab7f5d0c738a9488b772
Story: 2006409
Task: 36295
2019-08-28 15:15:53 +00:00
Stig Telfer
c6263dbfd4 Add support for Arista switches
Support for invoking Ansible EOS network modules for Arista switches.

Change-Id: I3a570adf43e1addce5eeab88e29ae4ded44669f0
2019-08-22 16:20:40 +02:00
Zuul
81912ab534 Merge "Fix inspector iPXE boot with UEFI compute nodes" 2019-08-21 22:00:56 +00:00
Zuul
51df5abb6e Merge "Update packages in virtualenvs" 2019-08-21 19:31:38 +00:00
Zuul
2638f83d08 Merge "Update release notes for stable/stein" 2019-08-21 14:14:04 +00:00
Mark Goddard
cd973178ea Update release notes for stable/stein
Change-Id: Ifc5a36743c98f32a9d841636119bc0d33621cef3
2019-08-16 13:32:19 +00:00
Pierre Riteau
2194bc2b5f Fix formatting of lists of commands in release notes
Without markup to generate a list, multiple commands are rendered on the
same line.

Change-Id: Ibc6d55c653a3d3f6523af42e7960cfc649989c9d
2019-08-16 15:17:14 +02:00
Zuul
9317024930 Merge "Generate openrc files during control host bootstrap" 2019-08-15 22:13:16 +00:00
Mark Goddard
ddfd6b6202 Update packages in virtualenvs
Kayobe uses a number of virtual environments on the remote hosts for
python dependencies such as shade, python-openstackclient, docker, etc.
By default these are stored in /opt/kayobe/venvs/. Typically we do not
provide version restrictions when installing these packages, so over the
course of time they may become stale and incompatible.

This change installs the latest version of packages allowed by OpenStack
upper constraints.

It also adds a new variable, 'pip_upper_constraints_file', to set the
upper constraints file. The existing variable
'kolla_upper_constraints_file' now defaults to the value of
'pip_upper_constraints_file'.

Change-Id: I8d2956f95bbc44b5a9e88e7569372048a62f12f5
Story: 2005923
Task: 34193
2019-08-15 11:01:49 +00:00
Zuul
fa4d345842 Merge "Remove storage management network from controllers" 2019-08-14 19:15:34 +00:00
Zuul
49b3dc66af Merge "Stop allocating network and broadcast addresses" 2019-08-14 19:12:46 +00:00
Zuul
90e713fd67 Merge "Add support for custom CloudKitty configuration" 2019-08-14 19:12:41 +00:00
Mark Goddard
9d01cf39b7 Generate openrc files during control host bootstrap
Fixes an issue where the admin-openrc.sh and public-openrc.sh files
would not be generated when preparing a new control host environment for
an existing cloud. These files are now generated during 'kayobe control
host bootstrap' if the Kolla Ansible 'passwords.yml' file exists in the
Kayobe configuration.

Change-Id: I47cc95bc4c4198532c8cfd2c105f1c7033e7b932
Story: 2001667
Task: 6713
2019-08-14 16:52:48 +00:00
Pierre Riteau
f5490793fb Remove storage management network from controllers
There is no need to connect controllers nodes to the storage management
network by default. It should only be used by storage nodes.

Change-Id: I6fa7640658fd112aaddbe08c4af01711fb1f7916
Story: 2006273
Task: 35973
2019-08-06 14:13:41 +01:00
Mark Goddard
c44f27b4ae Move to openstackdocstheme for docs & release notes
Sphinx configuration adapted from kolla ansible.

Change-Id: Ibcfbd9114a099d8b487f543d526b0cde0744d84a
2019-08-03 19:02:20 +01:00
Pierre Riteau
697e74e41a Stop allocating network and broadcast addresses
When an allocation pool range is not defined by the operator, we should
not include network and broadcast addresses in the list of IP addresses
to give to hosts.

Change-Id: Id6e14286b5eb2b767a515e7edfc56741fb8d2c78
Story: 2006267
Task: 35958
2019-08-02 14:08:34 +02:00
Pierre Riteau
ffb7f96e16 Add support for custom CloudKitty configuration
Change-Id: I0c0589387da53458f41df280dff0179715448aaa
Story: 2006292
Task: 36003
2019-07-27 15:58:43 +02:00
Pierre Riteau
57d7d276a3 Fix typo
Change-Id: I058d75af0236b8589667229a78b46b42aa3579b5
2019-07-23 16:20:18 +02:00
Mark Goddard
0ee55e19fa Fix inspector iPXE boot with UEFI compute nodes
The IPA ramdisk fails to boot on UEFI compute nodes with iPXE. We see
the following error on the console, followed by a kernel panic:

Fix kernel panic-not syncing: VFS: unable to mount root fs on unknown
block(0,0)

The cause is the lack of an initrd=ironic-agent.initramfs kernel
argument in inspector.ipxe.

The initrd argument is present in the kolla-ansible version of
inspector.ipxe, and now contains all parameters that we need (since
Rocky), so this change simply removes our custom template, and passes
through the extra kernel argument option configuration value.

Story: 2006214
Task: 35804

Story: 2006213
Task: 35803

Depends-On: https://review.opendev.org/670566
Change-Id: I8a103b7c6d3dd1f0433f922ae90deae9b92c0c78
2019-07-15 09:31:03 +01:00
Mark Goddard
650919f527 Disable chrony container by default
Fixes an issue where multiple NTP daemons could be running on the
overcloud hosts, due to Kolla Ansible deploying a chrony container by
default starting with the Rocky release.

Kayobe now overrides this default, to ensure that chrony does not conflict
with the NTP daemon deployed on the host. To use the containerised chrony
daemon instead, set ``kolla_enable_chrony`` to ``true`` in
``${KAYOBE_CONFIG_PATH}/kolla.yml``. This will also disable the host NTP
daemon.

To ensure that chrony is not running, Kayobe removes the chrony container
if ``kolla_enable_chrony`` is ``false`` in the following commands:

* ``kayobe overcloud service deploy``
* ``kayobe overcloud service reconfigure``
* ``kayobe overcloud service upgrade``

The play in Kayobe is tagged with ``stop-chrony``.

Change-Id: I89a973c0b600abece79bddcba5a46cc28a4f1df9
Story: 2005272
Task: 30122
2019-07-15 09:29:25 +01:00
Zuul
590f5ca885 Merge "Add command to update packages on the seed hypervisor host" 2019-07-12 21:28:44 +00:00
Zuul
85330b9166 Merge "Add support for custom Ceilometer configuration" 2019-07-12 21:28:40 +00:00
Zuul
8076acdb74 Merge "Adds support for Ansible 2.7, bump mimimum to 2.5" 2019-07-12 21:28:37 +00:00
Pierre Riteau
4667fd4e54 Add support for overriding OpenStack configuration globally
This commit allows Kayobe to generate a global.conf file for use by
Kolla Ansible to override configuration across all OpenStack services.

Change-Id: I6d144a945e1cde06fa9fdd03c30102458c0c9f8d
Story: 2005904
Task: 34153
2019-07-11 16:29:14 +00:00
Zuul
2002cf6019 Merge "Support customising refactored HAProxy config" 2019-07-08 16:17:11 +00:00
Zuul
3e99fa2f31 Merge "Add support for custom Gnocchi configuration" 2019-07-03 19:13:07 +00:00
Mark Goddard
405ccdada2 Adds support for Ansible 2.7, bump mimimum to 2.5
Updates the minimum version of Ansible from 2.4 to 2.5, and the maximum
supported version from 2.6 to 2.7.

Change-Id: If8071a9b5c85e5e69fbb333e91c84d10c20d80f3
Story: 2006143
Task: 35639
2019-07-02 10:38:54 +01:00
Pierre Riteau
77170a8547 Support customising refactored HAProxy config
Change-Id: I435ab9f88f73a73d7dc02f604f05e6f6def02eb5
2019-07-01 19:51:28 +01:00
Pierre Riteau
58f856fd00 Add support for custom Ceilometer configuration
Change-Id: I0a9e1a3b878231f9ce7bf295b5526c9cfabbbf9e
Story: 2005811
Task: 33551
2019-06-25 10:44:57 +01:00
Zuul
60c7a219cb Merge "Use BatchMode to check whether host is accessible via SSH" 2019-06-24 19:05:26 +00:00
Pierre Riteau
0e6703a322 Use BatchMode to check whether host is accessible via SSH
If password authentication is not disabled on the server, we may be
prompted for a password if the Kayobe Ansible user doesn't exist or
isn't configured for passwordless authentication.

By using BatchMode, the ssh command can fail quickly and proceed to
create the Kayobe Ansible user using the bootstrap user.

Change-Id: If22ed34dc4b6e87f8cf76c302948c955bddf2bc5
2019-06-11 16:01:03 +01:00
Pierre Riteau
8e69fb0ae8 Add command to update packages on the seed hypervisor host
Change-Id: I6edda0527a892c55261cc6fc48a0feb899b348d6
2019-06-07 16:36:28 +01:00
Pierre Riteau
49d4d7be79 Add support for custom Blazar configuration
Change-Id: I00ee10bdc7f8c1b6ab8576115c757e27b6c2c397
Story: 2005809
Task: 33549
2019-05-31 16:55:10 +01:00
Pierre Riteau
603527c215 Add support for custom Gnocchi configuration
Change-Id: I06c07108693766fd436a60a39f270c116931941e
Story: 2005810
Task: 33550
2019-05-31 16:29:54 +01:00
Mark Goddard
e96179921f Add commands for database backup and recovery
Uses the support [1] for database backups added to Kolla Ansible in the
Stein release.

Adds the following new commands:

kayobe overcloud database backup [--incremental]
kayobe overcloud database recover [--force-recovery-host <host>]

The recovery command is for recovering a clustered database that has
lost quorum.

[1] https://docs.openstack.org/kolla-ansible/latest/admin/mariadb-backup-and-restore.html

Change-Id: Ie16354cd01ea7dd83cd3d4058dd8451b8387600b
Story: 2005015
Task: 29493
2019-05-23 10:48:04 +01:00
Ross Martyn
d8ee0fa027 Multiple updates for Kayobe
---

1. Gather facts for localhost in kolla-ansible.yml
2. Don't include unconfigured networks in network_interfaces
3. Added Keystone configuration extra config merge

---

1. Facts are necessary for the kolla-ansible role,
which references the ansible_user_uid fact

2. It is possible to skip configuring a network,
by setting its name to None

This is done in networks.yml as follows:

admin_oc_net_name:

Currently, these networks may still be included in the 'network_interfaces'
list for each host, despite the fact that they are not in use.

A classic example is when ironic is not enabled, it is currently still
necessary to define provisioning and cleaning networks.

This change avoids including any networks that have their name set to None
in network_interfaces.

3. Added support for Keystone custom configuration

Added tests and documentation to add support for keystone extra configuration

Co-Authored-By: Mark Goddard <mark@stackhpc.com

Change-Id: Iaa304221b8093ac71f9cdbb23edc84d1517578da
2019-05-20 15:30:07 +01:00
Zuul
c020751a5d Merge "Support for Ceph and Swift storage networks, and improvements to Swift" 2019-05-15 13:48:05 +00:00
Mark Goddard
c6425d0898 Support arbitrarily named Docker drivers
Currently we configure daemon.json to use either devicemapper or overlay
as the storage driver. We should simply pass through whatever is
configured for docker_storage_driver.

Change-Id: Id4423030b5483fe4ecd4f324bc25800e5444fd63
Story: 2005649
Task: 30932
2019-05-13 13:52:20 +01:00
Scott Solkhon
6496cfc0ba Support for Ceph and Swift storage networks, and improvements to Swift
In a deployment that has both Ceph or Swift deployed it can be useful to seperate the network traffic.
This change adds support for dedicated storage networks for both Ceph and Swift. By default, the storage hosts are
attached to the following networks:

* Overcloud admin network
* Internal network
* Storage network
* Storage management network

This adds four additional networks, which can be used to seperate the storage network traffic as follows:

* Ceph storage network (ceph_storage_net_name) is used to carry Ceph storage
  data traffic. Defaults to the storage network (storage_net_name).
* Ceph storage management network (ceph_storage_mgmt_net_name) is used to carry
  storage management traffic. Defaults to the storage management network
  (storage_mgmt_net_name).
* Swift storage network (swift_storage_net_name) is used to carry Swift storage data
  traffic. Defaults to the storage network (storage_net_name).
* Swift storage replication network (swift_storage_replication_net_name) is used to
  carry storage management traffic. Defaults to the storage management network
  (storage_mgmt_net_name).

This change also includes several improvements to Swift device management and ring generation.

The device management and ring generation are now separate, with device management occurring during
'kayobe overcloud host configure', and ring generation during a new command, 'kayobe overcloud swift rings generate'.

For the device management, we now use standard Ansible modules rather than commands for device preparation.
File system labels can be configured for each device individually.

For ring generation, all commands are run on a single host, by default a host in the Swift storage group.
A python script runs in one of the kolla Swift containers, which consumes an autogenerated YAML config file that defines
the layout of the rings.

Change-Id: Iedc7535532d706f02d710de69b422abf2f6fe54c
2019-04-24 12:40:20 +00:00
Zuul
272709d54a Merge "Add openstack_release and openstack_branch" 2019-04-15 15:13:16 +00:00
Mark Goddard
73063732aa Add openstack_release and openstack_branch
Adds two new variables, 'openstack_release' and 'openstack_branch',
in ${KAYOBE_CONFIG_PATH}/openstack.yml for setting the current OpenStack
release and branch in a single place.

Also uses the master branch for the molecule tox job.

Change-Id: I851a1e6eddc6738005c1813599445b38c0ae0d57
2019-04-15 11:41:07 +00:00
Zuul
0a2c13accb Merge "Remove inspector_manage_firewall variable" 2019-04-10 11:30:31 +00:00
Mark Goddard
c2a35ce211 Remove inspector_manage_firewall variable
This is supported in kolla-ansible via the ironic_inspector_pxe_filter
variable, which can be added to globals.yml. The default value for that
variable changed in the Stein release from 'iptables' to 'dnsmasq',
since the iptables filter does not work with Docker CE [1].

This change removes the inspector_manage_firewall variable.

This change also adds an iptables rule in CI tests to allow DHCP packets
to be forwarded, to ensure bare metal servers can be deployed.

[1] https://bugs.launchpad.net/kolla-ansible/+bug/1823044

Depends-On: https://review.openstack.org/649673
Change-Id: Idac6777b4d97fbd17698fc2086ceb068d7b2e326
Related-Bug: #1823044
2019-04-09 13:53:59 +01:00
Zuul
0a73482576 Merge "Add support for configuring software RAID arrays using mdadm" 2019-04-08 14:22:08 +00:00