openstack/openstack-ansible-os_keystone
Code Issues Proposed changes

Clean up role tests

Browse Source 
Now that we're using the general templates, we can slim down
the role test definitions. We can also remove tests which are
not being watched, or which are fundamentally broken. With
this we can also remove unused scripts/plays.

We do the following:

1. We remove the 'ssl' job, given that the person working on
   that is no longer doing so, and no-one else has picked up
   the work.
2. We remove the 'upgrade' job, given that it's been broken
   all cycle due to the way the job executes and we need to
   regroup to figure out another way to do it.
3. We promote the distro_install jobs, given that they are
   all now working.
4. We promote the centos apache/uwsgi functional test, given
   that it is working consistently.

Change-Id: I67e0d8f4ab52449a80adb5c479faefbc83617025
This commit is contained in:
Jesse Pretorius 2018-09-07 15:07:47 +01:00 committed by Jesse Pretorius (odyssey4me)
parent 49ddbfd635
commit 4238d5a4f4
13 changed files with 7 additions and 392 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
tests/ansible-role-requirements.yml
View File

@ -43,10 +43,6 @@
src: https://git.openstack.org/openstack/openstack-ansible-openstack_openrc
scm: git
version: master
- name: os_previous_keystone
src: https://git.openstack.org/openstack/openstack-ansible-os_keystone
scm: git
version: stable/queens
- name: os_tempest
src: https://git.openstack.org/openstack/openstack-ansible-os_tempest
scm: git

5
tests/ansible.cfg
View File

@ -1,5 +0,0 @@
[defaults]
host_key_checking = False
[ssh_connection]
control_path = /tmp/%%h-%%r

26
tests/templates/auth.json
View File

@ -1,26 +0,0 @@
{
"auth": {
"identity": {
"methods": [
"password"
],
"password": {
"user": {
"domain": {
"id": "default"
},
"password": "{{ keystone_auth_admin_password }}",
"name": "admin"
}
}
},
"scope": {
"project": {
"domain": {
"id": "default"
},
"name": "admin"
}
}
}
}

25
tests/templates/locustfile.py
View File

@ -1,25 +0,0 @@
import json
import locust
TOKEN_PATH = 'http://{{ external_lb_vip_address }}:5000/v3/auth/tokens'
class Task(locust.TaskSet):
@locust.task
def authenticate(self):
# authenticate for a token and validate it
with open('/opt/auth.json', 'r') as f:
body = json.loads(f.read())
headers = {'Content-Type': 'application/json'}
response = self.client.post(TOKEN_PATH, json=body, headers=headers)
token = response.headers.get('X-Subject-Token')
headers = {
'X-Subject-Token': token,
'X-Auth-Token': token
}
self.client.get(TOKEN_PATH, headers=headers)
class MyLocust(locust.HttpLocust):
task_set = Task

45
tests/test-benchmark-keystone-upgrade-results.yml
View File

@ -1,45 +0,0 @@
---
# Copyright 2017, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Playbook for deploying upgrade benchmarking
hosts: localhost
user: root
become: true
tasks:
- name: Kill upgrade benchmarking process
command: pkill locust
- name: Register upgrade benchmarking results
async_status: jid={{ locust_benchmark.ansible_job_id }}
register: locust_benchmark_summary
- name: Display upgrade benchmarking results
debug:
msg: "{{ locust_benchmark_summary.stderr }}"
- name: Parse upgrade benchmarking results
set_fact:
locust_benchmark_results: >
{%- set results = {} %}
{%- for line in locust_benchmark_summary.stderr.split('\n') if 'Total' in line %}
{%- set summary_line = line | regex_replace('^ Total\s+', '') | regex_replace('(%\))?\s+|\(', ',') %}
{%- set summary_numbers = summary_line.split(',') %}
{%- set _ = results.update(requests_count = summary_numbers[0]) %}
{%- set _ = results.update(failure_rate = summary_numbers[2]) %}
{%- endfor %}
{{- results -}}
- name: Verify that there were minimal failures during upgrade benchmarking
assert:
that:
- "locust_benchmark_results.requests_count > 0"
- "(locust_benchmark_results.failure_rate | int) < 1"
msg: Multiple requests must be made with a less than 1% overall failure rate.

43
tests/test-benchmark-keystone-upgrade.yml
View File

@ -1,43 +0,0 @@
---
# Copyright 2017, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Playbook for deploying upgrade benchmarking
hosts: localhost
user: root
become: true
tasks:
- name: Install locust
pip:
name: "{{ item }}"
with_items:
- locustio
- pyzmq
- name: Drop benchmarking config files
template:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
owner: "root"
group: "root"
mode: "0644"
with_items:
- { src: "auth.json" , dest: "/opt/auth.json" }
- { src: "locustfile.py" , dest: "/opt/locustfile.py" }
- name: Run locust
command: locust -f /opt/locustfile.py --no-web --clients=1 --only-summary --host={{ keystone_service_publicuri }} --logfile=/var/log/locust.log
async: 1000
poll: 0
register: locust_benchmark
vars_files:
- common/test-vars.yml

42
tests/test-install-previous-keystone.yml
View File

@ -1,42 +0,0 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Playbook for deploying previous keystone
hosts: keystone_all
user: root
gather_facts: true
pre_tasks:
- name: Set keystone_venv_tag fact
set_fact:
keystone_venv_tag: testing-previous
- name: Set keystone_messaging fact
set_fact:
keystone_messaging_enabled: "{{ groups['rabbitmq_all'] is defined }}"
- include: common/ensure-rabbitmq.yml
vhost_name: "{{ keystone_rabbitmq_vhost }}"
user_name: "{{ keystone_rabbitmq_userid }}"
user_password: "{{ keystone_rabbitmq_password }}"
when: groups['rabbitmq_all'] is defined
- include: common/create-grant-db.yml
db_name: "{{ keystone_galera_database }}"
db_password: "{{ keystone_container_mysql_password }}"
roles:
- role: "os_previous_keystone"
post_tasks:
- name: Reset keystone_venv_tag fact
set_fact:
keystone_venv_tag: testing
vars_files:
- common/previous/test-vars.yml

101
tests/test-keystone-upgrades.sh
View File

@ -1,101 +0,0 @@
#!/usr/bin/env bash
# Copyright 2016, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# WARNING:
# This file is use by all OpenStack-Ansible roles for testing purposes.
# Any changes here will affect all OpenStack-Ansible role repositories
# with immediate effect.
# PURPOSE:
# This script executes test Ansible playbooks required for performing
# an upgrade test of the os_neutron role.
# Due to the way Ansible caches and handles modules, we need to run
# separate Ansible runs to ensure the "upgrade" uses the new
# "neutron_migrations_facts" module, instead of the cached version
# used when deploying the previous Neutron version.
## Shell Opts ----------------------------------------------------------------
set -e
## Vars ----------------------------------------------------------------------
export WORKING_DIR=${WORKING_DIR:-$(pwd)}
export ROLE_NAME=${ROLE_NAME:-''}
export ANSIBLE_PARAMETERS=${ANSIBLE_PARAMETERS:-"-vv"}
export TEST_PLAYBOOK=${TEST_PLAYBOOK:-$WORKING_DIR/tests/test-upgrade-pre.yml}
export TEST_CHECK_MODE=${TEST_CHECK_MODE:-false}
export TEST_IDEMPOTENCE=${TEST_IDEMPOTENCE:-false}
export COMMON_TESTS_PATH="${WORKING_DIR}/tests/common"
echo "ANSIBLE_OVERRIDES: ${ANSIBLE_OVERRIDES}"
echo "ANSIBLE_PARAMETERS: ${ANSIBLE_PARAMETERS}"
echo "TEST_PLAYBOOK: ${TEST_PLAYBOOK}"
echo "TEST_CHECK_MODE: ${TEST_CHECK_MODE}"
echo "TEST_IDEMPOTENCE: ${TEST_IDEMPOTENCE}"
## Functions -----------------------------------------------------------------
function execute_ansible_playbook {
export ANSIBLE_CLI_PARAMETERS="${ANSIBLE_PARAMETERS} -e @${ANSIBLE_OVERRIDES}"
export ANSIBLE_BIN=${ANSIBLE_BIN:-"ansible-playbook"}
CMD_TO_EXECUTE="${ANSIBLE_BIN} ${TEST_PLAYBOOK} $@ ${ANSIBLE_CLI_PARAMETERS}"
echo "Executing: ${CMD_TO_EXECUTE}"
echo "With:"
echo " ANSIBLE_INVENTORY: ${ANSIBLE_INVENTORY}"
echo " ANSIBLE_LOG_PATH: ${ANSIBLE_LOG_PATH}"
${CMD_TO_EXECUTE}
}
## Main ----------------------------------------------------------------------
# Ensure that the Ansible environment is properly prepared
source "${COMMON_TESTS_PATH}/test-ansible-env-prep.sh"
# Prepare environment for the initial deploy of (previous and current) Keystone
# No upgrading or testing is done yet.
export TEST_PLAYBOOK="${WORKING_DIR}/tests/test-upgrade-pre.yml"
export ANSIBLE_LOG_PATH="${ANSIBLE_LOG_DIR}/ansible-execute-keystone-infrainstall.log"
# Execute the setup of the Keystone environment
execute_ansible_playbook
# Prepare environment for the deploy of previous Keystone:
# No upgrading or testing is done yet.
export TEST_PLAYBOOK="${WORKING_DIR}/tests/test-install-previous-keystone.yml"
export ANSIBLE_LOG_PATH="${ANSIBLE_LOG_DIR}/ansible-execute-previous_keystone-install.log"
export PREVIOUS_VENV="ansible-previous"
export ANSIBLE_BIN="${WORKING_DIR}/.tox/${PREVIOUS_VENV}/bin/ansible-playbook"
source ${COMMON_TESTS_PATH}/test-create-previous-venv.sh
# Execute the setup of previous Keystone
execute_ansible_playbook
# Unset previous branch overrides
unset PREVIOUS_VENV
unset ANSIBLE_BIN
# Prepare environment for the upgrade of Keystone
export TEST_PLAYBOOK="${WORKING_DIR}/tests/test-upgrade-post.yml"
export ANSIBLE_LOG_PATH="${ANSIBLE_LOG_DIR}/ansible-execute-keystone-upgrade.log"
# Excute the upgrade of Keystone including testing/benchmarking
execute_ansible_playbook

32
tests/test-upgrade-post.yml
View File

@ -1,32 +0,0 @@
---
# Copyright 2017, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This playbook handles the installation of current Keystone
# Benchmarking, and tempest testing.
# Install upgrade benchmarking
- include: test-benchmark-keystone-upgrade.yml
# Install Keystone
- include: common/test-install-keystone.yml
# Install and execute Tempest
- include: common/test-install-tempest.yml
# Test Keystone
- include: test-keystone-functional.yml
# Test upgrade benchmarking results
- include: test-benchmark-keystone-upgrade-results.yml

35
tests/test-upgrade-pre.yml
View File

@ -1,35 +0,0 @@
---
# Copyright 2015, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This playbook handles the deployment of the Infrastructure
# And the previous version of keystone.
# Prepare the user ssh keys
- include: common/test-prepare-keys.yml
# Prepare the host
- include: common/test-prepare-host.yml
# Prepare the containers
- include: common/test-prepare-containers.yml
# Install haproxy
- include: common/test-install-haproxy.yml
# Install RabbitMQ/MariaDB
- include: common/test-install-infra.yml
# Ensure the repo is setup for previous version
- include: common/previous/test-repo-setup.yml

17
tox.ini
View File

@ -101,15 +101,6 @@ commands =
bash -c "{toxinidir}/tests/common/test-ansible-functional.sh"
[testenv:upgrade]
setenv =
{[testenv]setenv}
ANSIBLE_OVERRIDES={toxinidir}/tests/os_keystone-overrides.yml
CLONE_UPGRADE_TESTS=yes
commands =
bash -c "{toxinidir}/tests/test-keystone-upgrades.sh"
[testenv:uw_apache]
setenv =
{[testenv]setenv}
@ -118,14 +109,6 @@ commands =
bash -c "{toxinidir}/tests/common/test-ansible-functional.sh"
[testenv:ssl]
setenv =
{[testenv]setenv}
ANSIBLE_PARAMETERS=-vvv -e galera_use_ssl=True
commands =
bash -c "{toxinidir}/tests/common/test-ansible-functional.sh"
[testenv:linters]
commands =
bash -c "{toxinidir}/tests/common/test-ansible-env-prep.sh"

10
zuul.d/jobs.yaml
View File

@ -25,14 +25,6 @@
nodeset: ubuntu-xenial
- job:
name: openstack-ansible-uw_apache-centos-7-nv
name: openstack-ansible-uw_apache-centos-7
parent: openstack-ansible-uw_apache
voting: false
nodeset: centos-7
- job:
name: openstack-ansible-keystone-ssl-nv
parent: openstack-ansible-functional-ubuntu-xenial
voting: false
vars:
tox_env: ssl

14
zuul.d/project.yaml
View File

@ -20,17 +20,15 @@
- release-notes-jobs-python3
check:
jobs:
- openstack-ansible-upgrade-ubuntu-xenial
- openstack-ansible-uw_apache-centos-7-nv
- openstack-ansible-uw_apache-centos-7
- openstack-ansible-uw_apache-ubuntu-xenial
- openstack-ansible-keystone-ssl-nv
- openstack-ansible-functional-distro_install-ubuntu-bionic
# NOTE(hwoarang) Centos7 is having some troubles with repo dependencies
# so disabling until it's investigated.
- openstack-ansible-functional-distro_install-centos-7:
voting: false
- openstack-ansible-functional-distro_install-centos-7
- openstack-ansible-functional-distro_install-opensuse-423
gate:
jobs:
- openstack-ansible-upgrade-ubuntu-xenial
- openstack-ansible-uw_apache-centos-7
- openstack-ansible-uw_apache-ubuntu-xenial
- openstack-ansible-functional-distro_install-ubuntu-bionic
- openstack-ansible-functional-distro_install-centos-7
- openstack-ansible-functional-distro_install-opensuse-423