58 lines
2.0 KiB
Bash
Raw Normal View History

Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
#!/usr/bin/env bash
# Copyright 2014, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
## Shell Opts ----------------------------------------------------------------
set -e -u -x
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
## Library Check -------------------------------------------------------------
info_block "Checking for required libraries." 2> /dev/null || source $(dirname ${0})/scripts-library.sh
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
## Main ----------------------------------------------------------------------
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
info_block "Running Basic Ansible Lint Check"
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
# Install the development requirements.
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
if [ -f "dev-requirements.txt" ]; then
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
pip2 install -r dev-requirements.txt || pip install -r dev-requirements.txt
else
pip2 install ansible-lint || pip install ansible-lint
fi
# Run hacking/flake8 check for all python files
# Ignores the following rules due to how ansible modules work in general
# F403 'from ansible.module_utils.basic import *' used; unable to detect undefined names
# H303 No wildcard (*) import.
flake8 --ignore=F403,H303 $(grep -rln -e '^#!/usr/bin/env python' -e '^#!/bin/python' -e '^#!/usr/bin/python' * )
# Create keys if they don't already exist.
ssh_key_create
# Perform our simple sanity checks.
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
pushd playbooks
echo -e '[all]\nlocalhost ansible_connection=local' | tee local_only_inventory
# Do a basic syntax check on all playbooks and roles.
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
info_block "Running Syntax Check"
ansible-playbook -i local_only_inventory --syntax-check *.yml --list-tasks
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
# Perform a lint check on all playbooks and roles.
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
info_block "Running Lint Check"
ansible-lint --version
ansible-lint *.yml
popd