66 lines
2.4 KiB
Bash
Raw Normal View History

2014-08-26 18:08:15 -05:00
#!/usr/bin/env bash
# Copyright 2014, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
## Shell Opts ----------------------------------------------------------------
set -e -u +x
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
## Variables -----------------------------------------------------------------
export REPO_URL=${REPO_URL:-"https://github.com/stackforge/os-ansible-deployment.git"}
export REPO_BRANCH=${REPO_BRANCH:-"master"}
export WORKING_FOLDER=${WORKING_FOLDER:-"/opt/stackforge/os-ansible-deployment"}
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
# On normal AIO build the script should do everything possible to try for success
export MAX_RETRIES=${MAX_RETRIES:-"5"}
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
## Main ----------------------------------------------------------------------
# set verbosity
set -x
# install git so that we can fetch the repo
apt-get update && apt-get install -y git
2014-08-26 18:08:15 -05:00
# fetch the repo
Convert existing roles into galaxy roles This change implements the blueprint to convert all roles and plays into a more generic setup, following upstream ansible best practices. Items Changed: * All tasks have tags. * All roles use namespaced variables. * All redundant tasks within a given play and role have been removed. * All of the repetitive plays have been removed in-favor of a more simplistic approach. This change duplicates code within the roles but ensures that the roles only ever run within their own scope. * All roles have been built using an ansible galaxy syntax. * The `*requirement.txt` files have been reformatted follow upstream Openstack practices. * Dynamically generated inventory is now more organized, this should assist anyone who may want or need to dive into the JSON blob that is created. In the inventory a properties field is used for items that customize containers within the inventory. * The environment map has been modified to support additional host groups to enable the seperation of infrastructure pieces. While the old infra_hosts group will still work this change allows for groups to be divided up into seperate chunks; eg: deployment of a swift only stack. * The LXC logic now exists within the plays. * etc/openstack_deploy/user_variables.yml has all password/token variables extracted into the separate file etc/openstack_deploy/user_secrets.yml in order to allow seperate security settings on that file. Items Excised: * All of the roles have had the LXC logic removed from within them which should allow roles to be consumed outside of the `os-ansible-deployment` reference architecture. Note: * the directory rpc_deployment still exists and is presently pointed at plays containing a deprecation warning instructing the user to move to the standard playbooks directory. * While all of the rackspace specific components and variables have been removed and or were refactored the repository still relies on an upstream mirror of Openstack built python files and container images. This upstream mirror is hosted at rackspace at "http://rpc-repo.rackspace.com" though this is not locked to and or tied to rackspace specific installations. This repository contains all of the needed code to create and/or clone your own mirror. DocImpact Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk> Closes-Bug: #1403676 Implements: blueprint galaxy-roles Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
2015-02-14 10:06:50 -06:00
git clone -b ${REPO_BRANCH} ${REPO_URL} ${WORKING_FOLDER}
2014-08-26 18:08:15 -05:00
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
# run the same aio build script that is used in the OpenStack CI pipeline
cd ${WORKING_FOLDER}
bash scripts/gate-check-commit.sh
# put a motd in place to help the user know what stuff is accessible once the build is complete
cat > /etc/update-motd.d/20-openstack<< EOF
#!/usr/bin/env bash
echo ""
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
echo "############ os-ansible-deployment all-in-one build #############"
echo ""
echo " OpenStack Services are now listening on $(ip -o -4 addr show dev eth0 | awk -F '[ /]+' '/global/ {print $4}')"
echo ""
EOF
chmod +x /etc/update-motd.d/20-openstack
# put an motd in place to help the user know how to restart galera after reboot
cat > /etc/update-motd.d/21-galera<< EOF
#!/usr/bin/env bash
echo ""
echo "If this server has been rebooted, you will need to re-bootstrap"
echo "Galera to get the cluster operational. To do this execute:"
echo ""
echo "This requires you to identify the most advanced node. For details see http://galeracluster.com/documentation-webpages/quorumreset.html
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
echo ""
EOF
Implement local scripts for all gate tests This patch implements the following: - scripts-library.sh which includes commonly used functions, variables and other preparation commands for all other scripts - bootstrap-ansible.sh which only deploys a selected version of ansible and ensures that any other requirements are prepared on the deployment host - bootstrap-aio.sh which runs all host preparation actions for an all-in-one build - gate-check-lint.sh which runs a lint and syntax check - gate-check-commit.sh which runs all actions required for a gate commit check, utilising the other scripts where required - run-smoke-test.sh which runs tempest from inside the utility container - run-playbooks.sh which runs the playbooks - the existing conf.d/swift.yml is renamed to be an example configuration - the example configurations can be used as documentation - etc/network/interfaces.d/aio_interfaces.cfg, etc/rpc_deploy/conf.d/swift.yml and etc/rpc_deploy/rpc_user_config.yml are now configurations used for the AIO deployment - a workaround for https://bugs.launchpad.net/bugs/1244589 to ensure that DHCP checksums are implemented by the host which is required for the smoke tests to work - the removal of the rpc heat templates as they're unusable in their current state - setting MAX_RETRIES to 0, ensuring that any failures cause an immediate commit check failure in the gate - this prevents the masking of failures by retry attempts DocImpact Co-Authored-By: Kevin Carter <kevin.carter@rackspace.com> Closes-Bug: #1415883 Closes-Bug: #1417999 Closes-Bug: #1419807 Change-Id: I95242d48ad0fb055f16510803c8aa14dc183ac17
2015-02-04 15:51:06 +00:00
chmod +x /etc/update-motd.d/21-galera