Setup framework for IPv6-only deployment verification

Devstack IPv6 base job 'devstack-IPv6' setup the IPv6 setting
to deploy the services to listen on IPv6 address.

Tempest 'devstack-tempest-ipv6' job derived from 'devstack-IPv6'
job adds the playbooks to run the tests.

As part of Train community goal 'Support IPv6-Only Deployments',
we will expand the 'devstack-tempest-ipv6' job to do
IPv6-only deployments verification.

This commit define the new roles of 'ipv6-only-deployments-verification'
which will be invoked as part of run phase of 'devstack-tempest-ipv6' job.
This role will do IPv6-only setting and deployments verification via
tools/verify-ipv6-only-deployement.sh

IPv6 verification script can be extended further to perform more checks
and via project specific test case. Those tests will run as part of project
specific child jobs.

The 'devstack-tempest-ipv6' job will be used as parent for project specific
IPv6-only job. Those child job can extend the project specific IPv6 verification
by defining new playebook for post-run. That way the base verification
done in 'devstack-tempest-ipv6' will still run in addition to project specific
verificaiton and tests run.

Verification structure will be:
- 'devstack-IPv6' deploy the service on IPv6
- 'devstack-tempest-ipv6' run will verify the IPv6-only setting and listen address
- Child jobs derived from 'devstack-tempest-ipv6' will run the IPv6 related test case or
  any further IPv6 deployment verification.

This commit also adds the new job 'tempest-ipv6-only' which will run smoke
and ipv6 related tests present in Tempest. This job will be used to run
on 6 services (Nova, Neutron, Cinder, Keystone, Glance, Swift) deployed
by devstack.

Story: #2005477
Task: #35923

Change-Id: Ie1e5e5e02ee45ab7a4aae0f35bdb42afc2c3fe2e
This commit is contained in:
Ghanshyam 2019-07-17 09:46:41 +00:00 committed by Ghanshyam Mann
parent 504190d2ad
commit a0b8ca43e4
7 changed files with 178 additions and 31 deletions

View File

@ -7,12 +7,12 @@
This Tempest job provides the base for both the single and multi-node This Tempest job provides the base for both the single and multi-node
test setup. To run a multi-node test inherit from devstack-tempest and test setup. To run a multi-node test inherit from devstack-tempest and
set the nodeset to a multi-node one. set the nodeset to a multi-node one.
required-projects: required-projects: &base_required-projects
- opendev.org/openstack/tempest - opendev.org/openstack/tempest
timeout: 7200 timeout: 7200
roles: roles: &base_roles
- zuul: opendev.org/openstack/devstack - zuul: opendev.org/openstack/devstack
vars: vars: &base_vars
devstack_services: devstack_services:
tempest: true tempest: true
devstack_local_conf: devstack_local_conf:
@ -53,36 +53,36 @@
name: devstack-tempest-ipv6 name: devstack-tempest-ipv6
parent: devstack-ipv6 parent: devstack-ipv6
description: | description: |
Base Tempest IPv6 job. Base Tempest IPv6 job. This job is derived from 'devstack-ipv6'
required-projects: which set the IPv6-only setting for OpenStack services. As part of
- opendev.org/openstack/tempest run phase, this job will verify the IPv6 setting and check the services
endpoints and listen addresses are IPv6. Basically it will run the script
./tool/verify-ipv6-only-deployments.sh
Child jobs of this job can run their own set of tests and can
add post-run playebooks to extend the IPv6 verification specific
to their deployed services.
Check the wiki page for more details about project jobs setup
- https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing
required-projects: *base_required-projects
timeout: 7200 timeout: 7200
roles: roles: *base_roles
- zuul: opendev.org/openstack/devstack vars: *base_vars
vars: run: playbooks/devstack-tempest-ipv6.yaml
devstack_services:
tempest: true
devstack_local_conf:
test-config:
$TEMPEST_CONFIG:
compute:
min_compute_nodes: "{{ groups['compute'] | default(['controller']) | length }}"
test_results_stage_name: test_results
zuul_copy_output:
'{{ devstack_base_dir }}/tempest/etc/tempest.conf': logs
'{{ devstack_base_dir }}/tempest/etc/accounts.yaml': logs
'{{ devstack_base_dir }}/tempest/tempest.log': logs
'{{ stage_dir }}/{{ test_results_stage_name }}.subunit': logs
'{{ stage_dir }}/{{ test_results_stage_name }}.html': logs
'{{ stage_dir }}/stackviz': logs
extensions_to_txt:
conf: true
log: true
yaml: true
yml: true
run: playbooks/devstack-tempest.yaml
post-run: playbooks/post-tempest.yaml post-run: playbooks/post-tempest.yaml
- job:
name: tempest-ipv6-only
parent: devstack-tempest-ipv6
# This currently works from stable/pike on.
branches: ^(?!stable/ocata).*$
description: |
Integration test of IPv6-only deployments. This job runs
smoke and IPv6 relates tests only. Basic idea is to test
whether OpenStack Services listen on IPv6 addrress or not.
vars:
tox_envlist: ipv6-only
- job: - job:
name: tempest-full name: tempest-full
parent: devstack-tempest parent: devstack-tempest
@ -729,7 +729,7 @@
- tempest-multinode-full-py3: - tempest-multinode-full-py3:
irrelevant-files: *tempest-irrelevant-files irrelevant-files: *tempest-irrelevant-files
- tempest-tox-plugin-sanity-check: - tempest-tox-plugin-sanity-check:
irrelevant-files: irrelevant-files: &tempest-irrelevant-files-2
- ^.*\.rst$ - ^.*\.rst$
- ^doc/.*$ - ^doc/.*$
- ^etc/.*$ - ^etc/.*$
@ -738,6 +738,8 @@
- ^tempest/hacking/.*$ - ^tempest/hacking/.*$
- ^tempest/tests/.*$ - ^tempest/tests/.*$
# tools/ is not here since this relies on a script in tools/. # tools/ is not here since this relies on a script in tools/.
- tempest-ipv6-only:
irrelevant-files: *tempest-irrelevant-files-2
- tempest-slow: - tempest-slow:
irrelevant-files: *tempest-irrelevant-files irrelevant-files: *tempest-irrelevant-files
- tempest-slow-py3: - tempest-slow-py3:
@ -790,6 +792,8 @@
irrelevant-files: *tempest-irrelevant-files irrelevant-files: *tempest-irrelevant-files
- grenade-py3: - grenade-py3:
irrelevant-files: *tempest-irrelevant-files irrelevant-files: *tempest-irrelevant-files
- tempest-ipv6-only:
irrelevant-files: *tempest-irrelevant-files-2
experimental: experimental:
jobs: jobs:
- tempest-cinder-v2-api: - tempest-cinder-v2-api:

View File

@ -0,0 +1,24 @@
# Changes that run through devstack-tempest-ipv6 are likely to have an impact on
# the devstack part of the job, so we keep devstack in the main play to
# avoid zuul retrying on legitimate failures.
- hosts: all
roles:
- orchestrate-devstack
# We run tests only on one node, regardless how many nodes are in the system
- hosts: tempest
environment:
# This enviroment variable is used by the optional tempest-gabbi
# job provided by the gabbi-tempest plugin. It can be safely ignored
# if that plugin is not being used.
GABBI_TEMPEST_PATH: "{{ gabbi_tempest_path | default('') }}"
roles:
- setup-tempest-run-dir
- setup-tempest-data-dir
- acl-devstack-files
# Verify the IPv6-only deployments. This role will perform check for
# IPv6 only env for example Devstack IPv6 settings and services listen
# address is IPv6 etc. This is invoked before tests are run so that we can
# fail early if anything missing the IPv6 settings or deployments.
- ipv6-only-deployments-verification
- run-tempest

View File

@ -0,0 +1,16 @@
Verify the IPv6-only deployments
This role needs to be invoked from a playbook that
run tests. This role verifies the IPv6 setting on
devstack side and devstack deploy services on IPv6.
This role is invoked before tests are run so that
if any missing IPv6 setting or deployments can fail
the job early.
**Role Variables**
.. zuul:rolevar:: devstack_base_dir
:default: /opt/stack
The devstack base directory.

View File

@ -0,0 +1 @@
devstack_base_dir: /opt/stack

View File

@ -0,0 +1,4 @@
- name: Verify the ipv6-only deployments
become: true
become_user: stack
shell: "{{ devstack_base_dir }}/tempest/tools/verify-ipv6-only-deployments.sh"

View File

@ -0,0 +1,86 @@
#!/bin/bash
#
#
# NOTE(gmann): This script is used in 'devstack-tempest-ipv6' zuul job to verify that
# services are deployed on IPv6 properly or not. This will capture if any devstck or devstack
# plugins are missing the required setting to listen on IPv6 address. This is run as part of
# run phase of zuul job and before test run. Child job of 'devstack-tempest-ipv6'
# can expand the IPv6 verification specific to project by defining the new post-run script which
# will run along with this base script.
# If there are more common verification for IPv6 then we can always extent this script.
# Keep track of the DevStack directory
TOP_DIR=$(cd $(dirname "$0")/../../devstack && pwd)
source $TOP_DIR/stackrc
source $TOP_DIR/openrc admin admin
function verify_devstack_ipv6_setting {
local _service_host=$(echo $SERVICE_HOST | tr -d [])
local _host_ipv6=$(echo $HOST_IPV6 | tr -d [])
local _service_listen_address=$(echo $SERVICE_LISTEN_ADDRESS | tr -d [])
local _service_local_host=$(echo $SERVICE_LOCAL_HOST | tr -d [])
if [[ "$SERVICE_IP_VERSION" != 6 ]]; then
echo $SERVICE_IP_VERSION "SERVICE_IP_VERSION is not set to 6 which is must for devstack to deploy services with IPv6 address."
exit 1
fi
is_service_host_ipv6=$(python3 -c 'import oslo_utils.netutils as nutils; print(nutils.is_valid_ipv6("'$_service_host'"))')
if [[ "$is_service_host_ipv6" != "True" ]]; then
echo $SERVICE_HOST "SERVICE_HOST is not ipv6 which means devstack cannot deploy services on IPv6 address."
exit 1
fi
is_host_ipv6=$(python3 -c 'import oslo_utils.netutils as nutils; print(nutils.is_valid_ipv6("'$_host_ipv6'"))')
if [[ "$is_host_ipv6" != "True" ]]; then
echo $HOST_IPV6 "HOST_IPV6 is not ipv6 which means devstack cannot deploy services on IPv6 address."
exit 1
fi
is_service_listen_address=$(python3 -c 'import oslo_utils.netutils as nutils; print(nutils.is_valid_ipv6("'$_service_listen_address'"))')
if [[ "$is_service_listen_address" != "True" ]]; then
echo $SERVICE_LISTEN_ADDRESS "SERVICE_LISTEN_ADDRESS is not ipv6 which means devstack cannot deploy services on IPv6 address."
exit 1
fi
is_service_local_host=$(python3 -c 'import oslo_utils.netutils as nutils; print(nutils.is_valid_ipv6("'$_service_local_host'"))')
if [[ "$is_service_local_host" != "True" ]]; then
echo $SERVICE_LOCAL_HOST "SERVICE_LOCAL_HOST is not ipv6 which means devstack cannot deploy services on IPv6 address."
exit 1
fi
echo "Devstack is properly configured with IPv6"
echo "SERVICE_IP_VERSION: " $SERVICE_IP_VERSION "HOST_IPV6: " $HOST_IPV6 "SERVICE_HOST: " $SERVICE_HOST "SERVICE_LISTEN_ADDRESS: " $SERVICE_LISTEN_ADDRESS "SERVICE_LOCAL_HOST: " $SERVICE_LOCAL_HOST
}
function sanity_check_system_ipv6_enabled {
system_ipv6_enabled=$(python3 -c 'import oslo_utils.netutils as nutils; print(nutils.is_ipv6_enabled())')
if [[ $system_ipv6_enabled != "True" ]]; then
echo "IPv6 is disabled in system"
exit 1
fi
echo "IPv6 is enabled in system"
}
function verify_service_listen_address_is_ipv6 {
local endpoints_verified=False
local all_ipv6=True
endpoints=$(openstack endpoint list -f value -c URL)
for endpoint in ${endpoints}; do
local endpoint_address=$(echo "$endpoint" | awk -F/ '{print $3}' | awk -F] '{print $1}')
endpoint_address=$(echo $endpoint_address | tr -d [])
local is_endpoint_ipv6=$(python3 -c 'import oslo_utils.netutils as nutils; print(nutils.is_valid_ipv6("'$endpoint_address'"))')
if [[ "$is_endpoint_ipv6" != "True" ]]; then
all_ipv6=False
echo $endpoint ": This is not ipv6 endpoint which means corresponding service is not listening on IPv6 address."
continue
fi
endpoints_verified=True
done
if [[ "$all_ipv6" == "False" ]] || [[ "$endpoints_verified" == "False" ]]; then
exit 1
fi
echo "All services deployed by devstack is on IPv6 endpoints"
echo $endpoints
}
#First thing to verify if system has IPv6 enabled or not
sanity_check_system_ipv6_enabled
#Verify whether devstack is configured properly with IPv6 setting
verify_devstack_ipv6_setting
#Get all registrfed endpoints by devstack in keystone and verify that each endpoints address is IPv6.
verify_service_listen_address_is_ipv6

12
tox.ini
View File

@ -231,6 +231,18 @@ commands =
find . -type f -name "*.pyc" -delete find . -type f -name "*.pyc" -delete
tempest run --serial --regex '\[.*\bslow\b.*\]' {posargs} tempest run --serial --regex '\[.*\bslow\b.*\]' {posargs}
[testenv:ipv6-only]
envdir = .tox/tempest
sitepackages = {[tempestenv]sitepackages}
setenv = {[tempestenv]setenv}
deps = {[tempestenv]deps}
# Run only smoke and ipv6 tests. This env is used to tests
# the ipv6 deployments and basic tests run fine so that we can
# verify that services listen on IPv6 address.
commands =
find . -type f -name "*.pyc" -delete
tempest run --regex '\[.*\bsmoke|ipv6|test_network_v6\b.*\]' {posargs}
[testenv:venv] [testenv:venv]
deps = deps =
-c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master} -c{env:UPPER_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}