openstack
/
osc-placement
Code Issues Proposed changes

Use PlacementFixture in functional tests 

Change the functional tests to use the PlacementFixture instead
of devstack as the source of a placement API. This speeds up
the tests considerably and lowers the number of dependencies.

There are four primary changes:

* For each test a PlacementFixture is instantiated, using the
  usual in-RAM db and in-process placement.

* Because of some exceedingly confusing optimizations in
  osc_lib and python-openstackclient, done to improve start
  up time, a session to placement was caching the service
  url. This meant that after a first test succeeded, every
  subsequent one would not because it was trying to talk
  to a fake hostname that was no longer being intercepted.

  The workaround for this was to monkeypatch the method
  in the ClientCache class which provides access to a client (per
  service-type). The replacement method makes a new client
  every time.

* The previous tests would subprocess out to a real call of
  the openstack command and then interpret the results.

  Now, a run() method on OpenStackShell is called instead.
  This accepts arguments in the same way, but we need to
  a) capture stderr and stdout, b) make a try/except for
  SystemExit to get some error responses (mostly from
  the argparse lib which has a tendency to exit for you
  instead of politely telling you it wants to), c) deal
  with errors from commands ourself rather than using
  exceptions from the subprocess module.

  Switching to this form means that logging becomes in-process
  and more visible. To accomodate this the Capture fixture
  from placement is used. This was chosen because we are already
  pulling in the PlacementFixture. If this seems icky, I can fix
  it with a local one. This was the shorter path.

* The legacy dsvm jobs have been removed in favor of "standard"
  functional jobs for 2.7 and 3.6 that require openstack/placement.
  The playbooks associated with the legacy jobs are removed.

  tox.ini is adjusted to reflect this new setup. Because tox-siblings
  functional is being used, we don't want to share tox envs with the unit
  tests. The 3.5 functional job is removed because we no longer target
  that.

After running these for a while it became clear that there were
intermittent failures being caused by subunit attachments being too
large. This was eventually traced back to logging from all packages
being set to DEBUG even when something else was requested. That was
traced back to a poor interaction between the way that osc does
logging and the way oslo_logging does logging (used by placement and
thus the placement fixture). The workaround, embodied in the
RESET_LOGGING list in osc_placement/tests/functional/base.py, is to
get and reset the log level for a subset of the packages that are
used.

Change-Id: I7deda200b372ff6a7ba67b0c4fa0e53c4fa16ffc
Story: 2005411
Task: 30428
This commit is contained in:
Chris Dent 2019-04-11 23:17:17 +01:00
parent f79dbf7277
commit da8cd4d68b
13 changed files with 167 additions and 425 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

39
.zuul.yaml
View File

@ -1,26 +1,3 @@
- job:
name: osc-placement-dsvm-functional
parent: legacy-dsvm-base
run: playbooks/legacy/osc-placement-dsvm-functional/run.yaml
post-run: playbooks/legacy/osc-placement-dsvm-functional/post.yaml
timeout: 7200
required-projects:
- openstack/devstack-gate
- openstack/osc-placement
- job:
name: osc-placement-dsvm-functional-py3
parent: legacy-dsvm-base
description: |
Runs the osc-placement functional tests in a python 3 devstack
environment.
run: playbooks/legacy/osc-placement-dsvm-functional-py3/run.yaml
post-run: playbooks/legacy/osc-placement-dsvm-functional-py3/post.yaml
timeout: 7200
required-projects:
- openstack/devstack-gate
- openstack/osc-placement
- project:
templates:
- openstack-python-jobs
@ -32,9 +9,17 @@
- release-notes-jobs-python3
check:
jobs:
- osc-placement-dsvm-functional
- osc-placement-dsvm-functional-py3
- openstack-tox-functional:
required-projects:
- openstack/placement
- openstack-tox-functional-py36:
required-projects:
- openstack/placement
gate:
jobs:
- osc-placement-dsvm-functional
- osc-placement-dsvm-functional-py3
- openstack-tox-functional:
required-projects:
- openstack/placement
- openstack-tox-functional-py36:
required-projects:
- openstack/placement

124
osc_placement/tests/functional/base.py
View File

@ -10,14 +10,35 @@
# License for the specific language governing permissions and limitations
# under the License.
import logging
import random
import six
import subprocess
import fixtures
import six
from openstackclient import shell
from oslotest import base
from placement.tests.functional.fixtures import capture
from placement.tests.functional.fixtures import placement
import simplejson as json
# A list of logger names that will be reset to a log level
# of WARNING. Due (we think) to a poor interaction between the
# way osc does logging and oslo.logging, all packages are producing
# DEBUG logs. This results in test attachments (when capturing logs)
# that are sometimes larger than subunit.parser can deal with. The
# packages chosen here are ones that do not provide useful information.
RESET_LOGGING = [
'keystoneauth.session',
'oslo_policy.policy',
'placement.objects.trait',
'placement.objects.resource_class',
'placement.objects.resource_provider',
'oslo_concurrency.lockutils',
'osc_lib.shell',
]
RP_PREFIX = 'osc-placement-functional-tests-'
# argparse in python 2 and 3 have different error messages
@ -28,31 +49,77 @@ if six.PY3:
ARGUMENTS_REQUIRED = 'the following arguments are required: %s'
class CommandException(Exception):
def __init__(self, *args, **kwargs):
super(CommandException, self).__init__(args[0])
self.cmd = kwargs['cmd']
class BaseTestCase(base.BaseTestCase):
VERSION = None
@classmethod
def openstack(cls, cmd, may_fail=False, use_json=False):
result = None
try:
to_exec = ['openstack'] + cmd.split()
if use_json:
to_exec += ['-f', 'json']
if cls.VERSION is not None:
to_exec += ['--os-placement-api-version', cls.VERSION]
def setUp(self):
super(BaseTestCase, self).setUp()
self.useFixture(capture.Logging())
self.placement = self.useFixture(placement.PlacementFixture())
output = subprocess.check_output(to_exec, stderr=subprocess.STDOUT)
result = (output or b'').decode('utf-8')
except subprocess.CalledProcessError as e:
msg = 'Command: "%s"\noutput: %s' % (' '.join(e.cmd), e.output)
e.cmd = msg
# Work around needing to reset the session's notion of where
# we are going.
def mock_get(obj, instance, owner):
return obj.factory(instance)
# NOTE(cdent): This is fragile, but is necessary to work around
# the rather complex start up optimizations that are done in osc_lib.
# If/when osc_lib changes this will at least fail fast.
self.useFixture(fixtures.MonkeyPatch(
'osc_lib.clientmanager.ClientCache.__get__',
mock_get))
# Reset log level on a set of packages. See comment on RESET_LOGGING
# assigment, above.
for name in RESET_LOGGING:
logging.getLogger(name).setLevel(logging.WARNING)
def openstack(self, cmd, may_fail=False, use_json=False):
to_exec = []
# Make all requests as a noauth admin user.
to_exec += [
'--os-url', self.placement.endpoint,
'--os-token', self.placement.token,
]
if self.VERSION is not None:
to_exec += ['--os-placement-api-version', self.VERSION]
to_exec += cmd.split()
if use_json:
to_exec += ['-f', 'json']
# Context manager here instead of setUp because we only want
# output trapping around the run().
self.output = six.StringIO()
self.error = six.StringIO()
stdout_fix = fixtures.MonkeyPatch('sys.stdout', self.output)
stderr_fix = fixtures.MonkeyPatch('sys.stderr', self.error)
with stdout_fix, stderr_fix:
try:
os_shell = shell.OpenStackShell()
return_code = os_shell.run(to_exec)
# Catch SystemExit to trap some error responses, mostly from the
# argparse lib which has a tendency to exit for you instead of
# politely telling you it wants to.
except SystemExit as exc:
return_code = exc.code
if return_code:
msg = 'Command: "%s"\noutput: %s' % (' '.join(to_exec),
self.error.getvalue())
if not may_fail:
raise
raise CommandException(msg, cmd=' '.join(to_exec))
if use_json and result:
return json.loads(result)
output = self.output.getvalue() + self.error.getvalue()
if use_json and output:
return json.loads(output)
else:
return result
return output
def rand_name(self, name='', prefix=None):
"""Generate a random name that includes a random number
@ -79,10 +146,9 @@ class BaseTestCase(base.BaseTestCase):
try:
func(*args, **kwargs)
self.fail('Command does not fail as required (%s)' % signature)
except subprocess.CalledProcessError as e:
except CommandException as e:
self.assertIn(
message, six.text_type(e.output),
message, six.text_type(e),
'Command "%s" fails with different message' % e.cmd)
def resource_provider_create(self,
@ -99,9 +165,9 @@ class BaseTestCase(base.BaseTestCase):
def cleanup():
try:
self.resource_provider_delete(res['uuid'])
except subprocess.CalledProcessError as exc:
except CommandException as exc:
# may have already been deleted by a test case
err_message = exc.output.decode('utf-8').lower()
err_message = six.text_type(exc).lower()
if 'no resource provider' not in err_message:
raise
self.addCleanup(cleanup)
@ -166,9 +232,9 @@ class BaseTestCase(base.BaseTestCase):
def cleanup(uuid):
try:
self.openstack('resource provider allocation delete ' + uuid)
except subprocess.CalledProcessError as exc:
except CommandException as exc:
# may have already been deleted by a test case
if 'not found' in exc.output.decode('utf-8').lower():
if 'not found' in six.text_type(exc).lower():
pass
self.addCleanup(cleanup, consumer_uuid)
@ -264,9 +330,9 @@ class BaseTestCase(base.BaseTestCase):
def cleanup():
try:
self.trait_delete(name)
except subprocess.CalledProcessError as exc:
except CommandException as exc:
# may have already been deleted by a test case
err_message = exc.output.decode('utf-8').lower()
err_message = six.text_type(exc).lower()
if 'http 404' not in err_message:
raise
self.addCleanup(cleanup)

48
osc_placement/tests/functional/hooks/post_test_hook.sh
View File

@ -1,48 +0,0 @@
#!/bin/bash -xe
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# This script is executed inside post_test_hook function in devstack gate.
TOX_ENV=${TOX_ENV:-functional}
function generate_testr_results {
if [ -f .stestr/0 ]; then
.tox/$TOX_ENV/bin/stestr last --subunit > $WORKSPACE/testrepository.subunit
mv $WORKSPACE/testrepository.subunit $BASE/logs/testrepository.subunit
/usr/os-testr-env/bin/subunit2html $BASE/logs/testrepository.subunit $BASE/logs/testr_results.html
gzip -9 $BASE/logs/testrepository.subunit
gzip -9 $BASE/logs/testr_results.html
chmod a+r $BASE/logs/testrepository.subunit.gz $BASE/logs/testr_results.html.gz
fi
}
export OSCPLACEMENT_DIR="$BASE/new/osc-placement"
sudo chown -R $USER:stack $OSCPLACEMENT_DIR
# Go to the osc-placement dir
cd $OSCPLACEMENT_DIR
# Run tests
echo "Running osc-placement functional test suite"
set +e
# Preserve env for OS_ credentials
source $BASE/new/devstack/openrc admin admin
tox -e $TOX_ENV
EXIT_CODE=$?
set -e
# Collect and parse result
generate_testr_results
exit $EXIT_CODE

11
osc_placement/tests/functional/test_allocation.py
View File

@ -10,9 +10,10 @@
# License for the specific language governing permissions and limitations
# under the License.
import subprocess
import uuid
import six
from osc_placement.tests.functional import base
@ -68,11 +69,11 @@ class TestAllocation(base.BaseTestCase):
def test_allocation_create_empty(self):
consumer_uuid = str(uuid.uuid4())
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_allocation_set,
consumer_uuid, [])
self.assertIn('At least one resource allocation must be specified',
exc.output.decode('utf-8'))
six.text_type(exc))
def test_allocation_delete(self):
consumer_uuid = str(uuid.uuid4())
@ -91,9 +92,9 @@ class TestAllocation(base.BaseTestCase):
consumer_uuid = str(uuid.uuid4())
msg = "No allocations for consumer '{}'".format(consumer_uuid)
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_allocation_delete, consumer_uuid)
self.assertIn(msg, exc.output.decode('utf-8'))
self.assertIn(msg, six.text_type(exc))
class TestAllocation18(base.BaseTestCase):

62
osc_placement/tests/functional/test_inventory.py
View File

@ -10,7 +10,7 @@
# License for the specific language governing permissions and limitations
# under the License.
import subprocess
import six
from osc_placement.tests.functional import base
@ -38,11 +38,11 @@ class TestInventory(base.BaseTestCase):
def test_inventory_show_not_found(self):
rp_uuid = self.rp['uuid']
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_show,
rp_uuid, 'VCPU')
self.assertIn('No inventory of class VCPU for {}'.format(rp_uuid),
exc.output.decode('utf-8'))
six.text_type(exc))
def test_inventory_delete(self):
rp_uuid = self.rp['uuid']
@ -50,18 +50,18 @@ class TestInventory(base.BaseTestCase):
self.resource_inventory_set(rp_uuid, 'VCPU=8')
self.resource_inventory_delete(rp_uuid, 'VCPU')
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_show,
rp_uuid, 'VCPU')
self.assertIn('No inventory of class VCPU for {}'.format(rp_uuid),
exc.output.decode('utf-8'))
six.text_type(exc))
def test_inventory_delete_not_found(self):
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_delete,
self.rp['uuid'], 'VCPU')
self.assertIn('No inventory of class VCPU found for delete',
exc.output.decode('utf-8'))
six.text_type(exc))
def test_delete_all_inventories(self):
# Negative test to assert command failure because
@ -75,9 +75,9 @@ class TestInventory(base.BaseTestCase):
class TestSetInventory(base.BaseTestCase):
def test_fail_if_no_rp(self):
exc = self.assertRaises(
subprocess.CalledProcessError,
base.CommandException,
self.openstack, 'resource provider inventory set')
self.assertIn(base.ARGUMENTS_MISSING, exc.output.decode('utf-8'))
self.assertIn(base.ARGUMENTS_MISSING, six.text_type(exc))
def test_set_empty_inventories(self):
rp = self.resource_provider_create()
@ -86,33 +86,33 @@ class TestSetInventory(base.BaseTestCase):
def test_fail_if_incorrect_resource(self):
rp = self.resource_provider_create()
# wrong format
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], 'VCPU')
self.assertIn('must have "name=value"', exc.output.decode('utf-8'))
exc = self.assertRaises(subprocess.CalledProcessError,
self.assertIn('must have "name=value"', six.text_type(exc))
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], 'VCPU==')
self.assertIn('must have "name=value"', exc.output.decode('utf-8'))
exc = self.assertRaises(subprocess.CalledProcessError,
self.assertIn('must have "name=value"', six.text_type(exc))
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], '=10')
self.assertIn('must be not empty', exc.output.decode('utf-8'))
exc = self.assertRaises(subprocess.CalledProcessError,
self.assertIn('must be not empty', six.text_type(exc))
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], 'v=')
self.assertIn('must be not empty', exc.output.decode('utf-8'))
self.assertIn('must be not empty', six.text_type(exc))
# unknown class
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], 'UNKNOWN_CPU=16')
self.assertIn('Unknown resource class', exc.output.decode('utf-8'))
self.assertIn('Unknown resource class', six.text_type(exc))
# unknown property
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], 'VCPU:fake=16')
self.assertIn('Unknown inventory field', exc.output.decode('utf-8'))
self.assertIn('Unknown inventory field', six.text_type(exc))
def test_set_multiple_classes(self):
rp = self.resource_provider_create()
@ -143,10 +143,10 @@ class TestSetInventory(base.BaseTestCase):
def test_set_known_and_unknown_class(self):
rp = self.resource_provider_create()
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_inventory_set,
rp['uuid'], 'VCPU=8', 'UNKNOWN=4')
self.assertIn('Unknown resource class', exc.output.decode('utf-8'))
self.assertIn('Unknown resource class', six.text_type(exc))
self.assertEqual([], self.resource_inventory_list(rp['uuid']))
def test_replace_previous_values(self):
@ -171,26 +171,26 @@ class TestSetInventory(base.BaseTestCase):
def test_fail_if_incorrect_parameters_set_class_inventory(self):
exc = self.assertRaises(
subprocess.CalledProcessError,
base.CommandException,
self.openstack, 'resource provider inventory class set')
self.assertIn(base.ARGUMENTS_MISSING, exc.output.decode('utf-8'))
self.assertIn(base.ARGUMENTS_MISSING, six.text_type(exc))
exc = self.assertRaises(
subprocess.CalledProcessError,
base.CommandException,
self.openstack, 'resource provider inventory class set fake_uuid')
self.assertIn(base.ARGUMENTS_MISSING, exc.output.decode('utf-8'))
self.assertIn(base.ARGUMENTS_MISSING, six.text_type(exc))
exc = self.assertRaises(
subprocess.CalledProcessError,
base.CommandException,
self.openstack,
('resource provider inventory class set '
'fake_uuid fake_class --total 5 --unknown 1'))
self.assertIn('unrecognized arguments', exc.output.decode('utf-8'))
self.assertIn('unrecognized arguments', six.text_type(exc))
# Valid RP UUID and resource class, but no inventory field.
rp = self.resource_provider_create()
exc = self.assertRaises(
subprocess.CalledProcessError, self.openstack,
base.CommandException, self.openstack,
'resource provider inventory class set %s VCPU' % rp['uuid'])
self.assertIn(base.ARGUMENTS_REQUIRED % '--total',
exc.output.decode('utf-8'))
six.text_type(exc))
def test_set_inventory_for_resource_class(self):
rp = self.resource_provider_create()

21
osc_placement/tests/functional/test_resource_provider.py
View File

@ -11,7 +11,6 @@
# under the License.
import operator
import subprocess
import uuid
import six
@ -43,9 +42,9 @@ class TestResourceProvider(base.BaseTestCase):
rp_uuid = six.text_type(uuid.uuid4())
msg = 'No resource provider with uuid ' + rp_uuid + ' found'
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_provider_delete, rp_uuid)
self.assertIn(msg, exc.output.decode('utf-8'))
self.assertIn(msg, six.text_type(exc))
def test_resource_provider_set(self):
orig_name = self.rand_name('test_rp_orig_name')
@ -65,9 +64,9 @@ class TestResourceProvider(base.BaseTestCase):
rp_uuid = six.text_type(uuid.uuid4())
msg = 'No resource provider with uuid ' + rp_uuid + ' found'
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_provider_set, rp_uuid, 'test')
self.assertIn(msg, exc.output.decode('utf-8'))
self.assertIn(msg, six.text_type(exc))
def test_resource_provider_show(self):
created = self.resource_provider_create()
@ -103,9 +102,9 @@ class TestResourceProvider(base.BaseTestCase):
rp_uuid = six.text_type(uuid.uuid4())
msg = 'No resource provider with uuid ' + rp_uuid + ' found'
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_provider_show, rp_uuid)
self.assertIn(msg, exc.output.decode('utf-8'))
self.assertIn(msg, six.text_type(exc))
def test_resource_provider_list(self):
rp1 = self.resource_provider_create()
@ -238,12 +237,12 @@ class TestResourceProvider114(base.BaseTestCase):
child = self.resource_provider_create(
parent_provider_uuid=parent1['uuid'])
exc = self.assertRaises(
subprocess.CalledProcessError,
base.CommandException,
self.resource_provider_set,
child['uuid'],
name='mandatory_name_2',
parent_provider_uuid=parent2['uuid'])
self.assertIn('HTTP 400', exc.output.decode('utf-8'))
self.assertIn('HTTP 400', six.text_type(exc))
def test_resource_provider_list_in_tree(self):
rp1 = self.resource_provider_create()
@ -262,10 +261,10 @@ class TestResourceProvider114(base.BaseTestCase):
parent = self.resource_provider_create()
self.resource_provider_create(parent_provider_uuid=parent['uuid'])
exc = self.assertRaises(
subprocess.CalledProcessError,
base.CommandException,
self.resource_provider_delete,
parent['uuid'])
self.assertIn('HTTP 409', exc.output.decode('utf-8'))
self.assertIn('HTTP 409', six.text_type(exc))
class TestResourceProvider118(base.BaseTestCase):

7
osc_placement/tests/functional/test_usage.py
View File

@ -11,9 +11,10 @@
# under the License.
import operator
import subprocess
import uuid
import six
from osc_placement.tests.functional import base
@ -46,12 +47,12 @@ class TestUsage(base.BaseTestCase):
def test_usage_not_found(self):
rp_uuid = str(uuid.uuid4())
exc = self.assertRaises(subprocess.CalledProcessError,
exc = self.assertRaises(base.CommandException,
self.resource_provider_show_usage,
rp_uuid)
self.assertIn(
'No resource provider with uuid {} found'.format(rp_uuid),
exc.output.decode('utf-8')
six.text_type(exc)
)
def test_usage_empty(self):

81
playbooks/legacy/osc-placement-dsvm-functional-py3/post.yaml
View File

@ -1,81 +0,0 @@
- hosts: primary
tasks:
# TODO(mriedem): Need to cleanup these tasks.
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=**/*nose_results.html
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=**/*testr_results.html.gz
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/.testrepository/tmp*
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=**/*testrepository.subunit.gz
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}/tox'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/.tox/*/log/*
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/logs/**
- --include=*/
- --exclude=*
- --prune-empty-dirs

49
playbooks/legacy/osc-placement-dsvm-functional-py3/run.yaml
View File

@ -1,49 +0,0 @@
- hosts: all
name: osc-placement-dsvm-functional-py3
tasks:
- name: Ensure legacy workspace directory
file:
path: '{{ ansible_user_dir }}/workspace'
state: directory
- shell:
cmd: |
set -e
set -x
cat > clonemap.yaml << EOF
clonemap:
- name: openstack/devstack-gate
dest: devstack-gate
EOF
/usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \
https://opendev.org \
openstack/devstack-gate
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'
- shell:
cmd: |
set -e
set -x
export PYTHONUNBUFFERED=true
export DEVSTACK_PROJECT_FROM_GIT=osc-placement
export DEVSTACK_GATE_USE_PYTHON3=True
export TOX_ENV=functional-py3
# Only enable keystone and placement to make devstack setup faster.
# We enable mysql since we need a database, but don't need rabbit
# or anything else really.
export OVERRIDE_ENABLED_SERVICES=key,placement-api,mysql
function post_test_hook {
# Configure and run functional tests
$BASE/new/osc-placement/osc_placement/tests/functional/hooks/post_test_hook.sh
}
export -f post_test_hook
cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh
./safe-devstack-vm-gate-wrap.sh
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'

80
playbooks/legacy/osc-placement-dsvm-functional/post.yaml
View File

@ -1,80 +0,0 @@
- hosts: primary
tasks:
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=**/*nose_results.html
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=**/*testr_results.html.gz
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/.testrepository/tmp*
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=**/*testrepository.subunit.gz
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}/tox'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/.tox/*/log/*
- --include=*/
- --exclude=*
- --prune-empty-dirs
- name: Copy files from {{ ansible_user_dir }}/workspace/ on node
synchronize:
src: '{{ ansible_user_dir }}/workspace/'
dest: '{{ zuul.executor.log_root }}'
mode: pull
copy_links: true
verify_host: true
rsync_opts:
- --include=/logs/**
- --include=*/
- --exclude=*
- --prune-empty-dirs

47
playbooks/legacy/osc-placement-dsvm-functional/run.yaml
View File

@ -1,47 +0,0 @@
- hosts: all
name: Autoconverted job legacy-osc-placement-dsvm-functional from old job gate-osc-placement-dsvm-functional-ubuntu-xenial-nv
tasks:
- name: Ensure legacy workspace directory
file:
path: '{{ ansible_user_dir }}/workspace'
state: directory
- shell:
cmd: |
set -e
set -x
cat > clonemap.yaml << EOF
clonemap:
- name: openstack/devstack-gate
dest: devstack-gate
EOF
/usr/zuul-env/bin/zuul-cloner -m clonemap.yaml --cache-dir /opt/git \
https://opendev.org \
openstack/devstack-gate
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'
- shell:
cmd: |
set -e
set -x
export PYTHONUNBUFFERED=true
export DEVSTACK_PROJECT_FROM_GIT=osc-placement
# Only enable keystone and placement to make devstack setup faster.
# We enable mysql since we need a database, but don't need rabbit
# or anything else really.
export OVERRIDE_ENABLED_SERVICES=key,placement-api,mysql
function post_test_hook {
# Configure and run functional tests
$BASE/new/osc-placement/osc_placement/tests/functional/hooks/post_test_hook.sh
}
export -f post_test_hook
cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh
./safe-devstack-vm-gate-wrap.sh
executable: /bin/bash
chdir: '{{ ansible_user_dir }}/workspace'
environment: '{{ zuul | zuul_legacy_vars }}'

1
test-requirements.txt
View File

@ -8,3 +8,4 @@ coverage>=4.0 # Apache-2.0
oslotest>=1.10.0 # Apache-2.0
python-openstackclient>=3.3.0 # Apache-2.0
stestr>=1.0.0 # Apache-2.0
wsgi-intercept>=1.7.0 # MIT License

22
tox.ini
View File

@ -17,37 +17,31 @@ setenv =
VIRTUAL_ENV={envdir}
PYTHONWARNINGS=ignore::DeprecationWarning:distutils,ignore::DeprecationWarning:site
PYTHONDONTWRITEBYTECODE=1
# NOTE(rpodolyaka): allow passing of keystone credentials via env variables
passenv = OS_*
deps =
-c{env:UPPER_CONSTRAINTS_FILE:https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt}
-r{toxinidir}/test-requirements.txt
-r{toxinidir}/requirements.txt
commands = stestr run {posargs}
# NOTE(cdent): Functional tests require a running openstack
# and OS_* variables in the environment to allow authentication.
# This is because the openstack client is called against a
# real cloud.
# NOTE(cdent): Do not set envdir here, for the sake of the
# NOTE(cdent): Do not set envdir here as it confuses tox-siblings.
# gate functional jobs, which use the 'functional' path when
# copying files.
[testenv:functional]
basepython = python2.7
deps =
{[testenv]deps}
git+https://opendev.org/openstack/placement.git#egg=openstack-placement
commands = stestr --test-path=./osc_placement/tests/functional run {posargs}
# Used by the python3 functional job in the gate.
[testenv:functional-py3]
commands =
{[testenv:functional]commands}
[testenv:functional-py36]
envdir = {toxworkdir}/py36
deps =
{[testenv:functional]deps}
commands =
{[testenv:functional]commands}
[testenv:functional-py37]
envdir = {toxworkdir}/py37
deps =
{[testenv:functional]deps}
commands =
{[testenv:functional]commands}