Added -g for showing groups per instance and -G for showing instances
per group.
Also fixed up some pep8 and pylint issues.
Closes-Bug: 1482141
Change-Id: I698384467268336ecc74f865ee35f3e39cad5186
This is a barebones start to the upgrade script for the Liberty cycle.
This change simply cleans it out, more work will come later.
Change-Id: I86515970164a000d321ac4375497bfeba860ba8e
Implements: blueprint liberty-upgrade-path
This change updates all fo the names that we were using to the post
openstack migration name for openstack-ansible.
Change-Id: I6524af53ed02e19a0f56908e42a65d2dae8b71e3
This should help find the slowest Ansible tasks that need to be optimized.
It is only enabled by default for gate jobs.
The profiling callback code is MIT licensed. Per legal-discuss, I've added
the MIT license to the LICENSE.txt at the root of the repository along with
a note about which code is MIT licensed.
Thread on legal-discuss:
http://lists.openstack.org/pipermail/legal-discuss/2015-September/000398.html
Closes-Bug: #1488639
Change-Id: I46a2c495d9f8f1131ec08be99696efc39133f291
Apt updates/installs are the most common cause of build timeouts
within HP Cloud gate checks.
This patch changes both the AIO and the gate check from using the
Rackspace Ubuntu mirror for everything to:
1) The AIO bootstrap will use whichever mirror is already defined
on the host operating system.
2) The gate check will use a known mirror which is defined per
cloud provider. If the cloud provider is not known, or the
script is being used outside of OpenStack-CI, then the gate
checkwill fall back to using the AIO's mechanism.
The idea is to make use of an apt mirror that is as close to the
instance as possible in order to speed up apt updates and installs.
Closes-Bug: #1491749
Change-Id: Ia37100bc198b64b74f27ea98ec5956e7cf084883
Disable scatter-gather offload on host bridges to eliminate
kernel traces that may impact container connectivity. Only
addressing AIO interfaces for now as host configuration for
actual deployments resides in documentation.
Change-Id: Ia66b2bb64b9ace66f5fa3ca8edcc9909af54a4f2
Partial-Bug: #1488815
Co-Authored-By: Evan Callicoat <apsu@propter.net>
In order to ensure that the ansible output in the jenkins console
log is output as it comes it is necessary to turn off the python
output buffering.
Change-Id: I0f8f11cd04fdfd6c26d5a5afc0e8647648edb1bb
When performing an upgrade, this project strives to have minimal
downtime for VMs that are running. By removing the apparmor profile as a
precondition for upgrades, when the container create role runs, the
profile will default to contained (the most restrictive profile). This
causes instance downtime since neutron can not create network
namespaces.
Related-bug: 1487130
Closes-bug: 1489144
Change-Id: Ife7aab044c7cb882a89c6b108b2d66f5e39aa10c
This patch removes the upstream repo dependency so that
all builds executing setup-infrastructure now execute a
local repository build instead of an rcync clone from
an upstream source.
Change-Id: Idf9beb9ea396d241758d4e970a4f88fef96cdf46
Implements: blueprint remove-upstream-repo-dependency
This patch sets the command to install the iptables-persistent
package in the run-playbooks script to append its output to the
existing log instead of overwriting it.
This is to ensure that outputs from both commands are captured,
not just the second one.
Change-Id: I4d45382cb44c5f811818fe1229a63e6a2c2ecdee
Partial-Bug: #1485917
This patch removes read only disks from the candidates to be used
to store the lxc and cinder lvm volume groups.
Change-Id: I8fc0842f3629ac3c9fd01b01ecf37ffc68e3b871
Closes-Bug: #1487055
This patch properly enables or disables Ceilometer, general
OpenStack and Swift deployment properly.
For the moment the containers will still be created, but
none of the related software will be installed, configured
or tested.
A fix to limit the containers created will be implemented
after a revision of the haproxy playbook/role to accommodate
this is implemented.
Closes-Bug: #1485945
Implements: blueprint split-aio-gates
Change-Id: Ia6657a02a6d1c53a4d76d7a17f74748ec9d2a2ee
In order to be more consistent, and also account for different
environments and platforms, this patch implements the use of
#!/usr/bin/env as the shebang for all bash scripts in the
environment.
Closes-Bug: #1485557
Change-Id: I93e6a47fd6be816620682eddb2880e38f11bf675
This patch enables debug logging for all OpenStack services
in the gate checks to ensure that the maximum amount of
information is available for debugging gate check failures.
Change-Id: Ide583fa6dc06008641bbc10b1abcaa816e662337
Closes-Bug: #1482572
Previously, we simply checked $? which at that point would be the exit
status of echo, not openstack-ansible. By recording the actual
openstack-ansible exit status, we can properly report failures of the
upgrade script.
Closes-bug: 1480342
Change-Id: Icf43bea84660e4160a2dfcdb4ac93055340b3573
(cherry picked from commit 8a106d184a)
When running in an AIO environment, we need to drop an iptables rule to
ensure that communication between instances and the neutron metadata
service works.
Change-Id: Icc081fe83712ce883baa88f99db60c52dcc4c1ae
Closes-Bug: #1483603
Enacting the log link creation and the ansible.cfg change has
resulted in polluted patch reviews by developers making use of
AIO's for dev/test purposes.
This patch moves the Ansible logging changes to the
gate-check-script only as that's the only time that it's
actually required.
Change-Id: I4a1accad94ae153bf363b53fda0905e814c15173
Closes-Bug: #1479824
This patch adds a small script that automates the process of accessing a
service provider (SP) cloud using credentials from a identity provider
cloud (IdP), where both clouds use Keystone based authentication. The
script performs the complete authentication flow and displays the token
and endpoints to use with the openstack command line client.
Implements: blueprint keystone-federation
Change-Id: I4b8113d0aef9c754fb55497d44138df660332bb8
This patch fixes the following:
1. Properly quote arguments to run_lock function
2. Properly parse out the playbook filename in run_lock
Specifically the upgrade steps where we were using
"-e 'rabbitmq_upgrade=true' setup-infrastructure.yml"
"/tmp/fix_container_interfaces.yml || true"
Were causing issues and this patch resolves them.
Closes-bug: 1479916
Change-Id: I809085d6da493f7f7d545547a0d984c0e7b1bf45
(cherry picked from commit 560fbbdb07)
Currently the playbooks do not allow Ceph to be configured as a backend
for Cinder, Glance or Nova. This commit adds a new role called
ceph_client to do the required configuration of the hosts and updates
the service roles to include the required configuration file changes.
This commit requires that a Ceph cluster already exists and does not
make any changes to that cluster.
ceph_client role, run on the OpenStack service hosts
- configures the Ceph apt repo
- installs any required Ceph dependencies
- copies the ceph.conf file and appropriate keyring file to /etc/ceph
- creates the necessary libvirt secrets
os_glance role
glance-api.conf will set the following variables for Ceph:
- [DEFAULT]/show_image_direct_url
- [glance_store]/stores
- [glance_store]/rbd_store_pool
- [glance_store]/rbd_store_user
- [glance_store]/rbd_store_ceph_conf
- [glance_store]/rbd_store_chunk_size
os_nova role
nova.conf will set the following variables for Ceph:
- [libvirt]/rbd_user
- [libvirt]/rbd_secret_uuid
- [libvirt]/images_type
- [libvirt]/images_rbd_pool
- [libvirt]/images_rbd_ceph_conf
- [libvirt]/inject_password
- [libvirt]/inject_key
- [libvirt]/inject_partition
- [libvirt]/live_migration_flag
os_cinder is not updated because ceph is defined as a backend and that
is generated from a dictionary of the config, for an example backend
config, see etc/openstack_deploy/openstack_user_config.yml.example
pw-token-gen.py is updated so that variables ending in uuid are assigned
a UUID.
DocImpact
Implements: blueprint ceph-block-devices
Closes-Bug: #1455238
Change-Id: Ie484ce0bbb93adc53c30be32f291aa5058b20028
This patch changes the number of forks used by ansible when
using any of the convenience (and thus gate check) scripts
to the number of processors available on the deployment
system.
The previous values used were found to cause ssh connection
errors and it was found that reducing the number improved
the chances of success.
This patch also removes the forks setting from ansible.conf
so that ansible will use the default value when run in any
other way. This leaves the decision of setting the number
of forks to the deployer, as it should be.
Change-Id: I31ad7353344f7994063127ecfce8f4733769234c
Closes-Bug: #1479812
The in-tree version of user_group_vars.yml was removed in
30f9443c5d, but the corresponding
reference in the upgrade script was not also updated.
This commit changes the behavior to remove the file from /etc/ if found.
Change-Id: I9f5b061289c5f43e32983845469f5123cc9f209d
Closes-Bug: #1479501
If mongodb has been installed as a backend for ceilometer (as it is by
the bootstrap-aio script), we want it removed in the teardown. This
commit updates the teardown.sh to do just that.
Also s/remote_pip_pacakges/remove_pip_packages/
Change-Id: I909feac8321feaf66f44642cac1b729ded10d2fb
This adds a forced apt update for packages and keys which is intended to help
with the HPB4 gate issues. This also creates logs for all ansible commands
run in the gate process.
Change-Id: I79b64567e615d83c52c1a8727082345ffe265c87
The rabbitmq playbook is designed to run in parallel across the cluster.
This causes an issue when upgrading rabbitmq to a new major or minor
version because RabbitMQ does not support doing an online migration of
datasets between major versions. while a minor release can be upgrade
while online it is recommended to bring down the cluster to do any
upgrade actions. The current configuration takes no account of this.
Reference:
https://www.rabbitmq.com/clustering.html#upgrading for further details.
* A new variable has been added called `rabbitmq_upgrade`. This is set to
false by default to prevent a new version being installed unintentionally.
To run the upgrade, which will shutdown the cluster, the variable can be
set to true on the commandline:
Example:
openstack-ansible -e rabbitmq_upgrade=true \
rabbitmq-install.yml
* A new variable has been added called `rabbitmq_ignore_version_state`
which can be set "true" to ignore the package and version state tasks.
This has been provided to allow a deployer to rerun the plays in an
environment where the playbooks have been upgraded and the default
version of rabbitmq has changed within the role and the deployer has
elected to upgraded the installation at that time. This will ensure a
deployer is able to recluster an environment as needed without
effecting the package state.
Example:
openstack-ansible -e rabbitmq_ignore_version_state=true \
rabbitmq-install.yml
* A new variable has been added `rabbitmq_primary_cluster_node` which
allows a deployer to elect / set the primary cluster node in an
environment. This variable is used to determine the restart order
of RabbitMQ nodes. IE this will be the last node down and first one
up in an environment. By default this variable is set to:
rabbitmq_primary_cluster_node: "{{ groups['rabbitmq_all'][0] }}"
scripts/run-upgrade.sh has been modified to pass 'rabbitmq_upgrade=true'
on the command line so that RabbitMQ can be upgraded as part of the
upgrade between OpenStack versions.
DocImpact
Change-Id: I17d4429b9b94d47c1578dd58a2fb20698d1fe02e
Closes-bug: #1474992
This patch allows a user to set the number of containers per host for
each type of container that has a default affinity greater than 1 for an
AIO build.
Currently this list is:
* keystone
* galera
* rabbit_mq
* horizon
* repo
For example, one could set NUM_keystone_CONTAINERS=1 before running an
AIO to ensure that only one keystone container is built.
Change-Id: If90704a50414150ca37584e1f823ba0e85e287ac
Partially-Implements: blueprint split-aio-gates
This change adds a container task to ensure that container networks are up
and using the new configs as written by the lxc-container-create play. This
should resolve an issue where the container networks could be in a down
state after an upgrade due to a configuration file change.
A run function was also added to make it possible for a deployer to know
where in the upgrade process something might have failed and the order in
which the tasks may need to be rerun to continue the upgrade.
Change-Id: If02c4e269375368b6f613c5a9e3c947dddbd27f9
Closes-Bug: #1474585
Partial-Bug: #1475727
The `successerator` has been re-enabled using a single retry which will also help
the gate process to ensure that we're not having to deal with so many transient
failures.
Change-Id: I7ca374ce18b7a78b0a1563d5244b1d9ac52c4d92
Currently mysql is only (re)started when creating a cluster, adding a
node or the service isn't running. This means changes to configuration
files are not used.
This commit moves the restart functionality to a handler and sets
notifies on the the appropriate tasks so that when the config is changes
mysql is restarted.
This commit modifies the galera play to run in serial so that only one
instance of mysql can be restarted at a time. This is to prevent the
possibility of them all being restarted in one go. An additional play
'Check galera cluster status' has been added, this makes an attempt to
check that the cluster is healthy before proceeding with any possible
config changes. The checks can be overridden by setting
'galera_ignore_cluster_state=true' when running the playbook.
A facts module has been added to gather information about the status of
the cluster for use by the galera-install playbook.
DocImpact
Closes-bug: #1449958
Change-Id: Id83ba642c114d1f5cac04cc219be151a82a1023f