Use bundletester for amulet test execution
Switch to using bundletester for execution of functional tests, leveraging tox to build out test virtualenvs. Rename amulet tests inline with gate-* and dev-* naming standards. Update README to refer to functional testing section of the charm guide. Add basic multi-unit test for Xenial. Change-Id: I06a976d9fc6418f04f46f3c3cca83d2fb47f3ee4
This commit is contained in:
parent
7cd6ab7e2b
commit
6578871698
3
Makefile
3
Makefile
|
@ -11,8 +11,7 @@ test:
|
|||
|
||||
functional_test:
|
||||
@echo Starting amulet tests...
|
||||
@tests/setup/00-setup
|
||||
@juju test -v -p AMULET_HTTP_PROXY,AMULET_OS_VIP --timeout 2700
|
||||
@tox -e func27
|
||||
|
||||
bin/charm_helpers_sync.py:
|
||||
@mkdir -p bin
|
||||
|
|
|
@ -7,3 +7,17 @@ flake8>=2.2.4,<=2.4.1
|
|||
os-testr>=0.4.1
|
||||
charm-tools>=2.0.0
|
||||
requests==2.6.0
|
||||
# BEGIN: Amulet OpenStack Charm Helper Requirements
|
||||
# Liberty client lower constraints
|
||||
amulet>=1.14.3,<2.0
|
||||
bundletester>=0.6.1,<1.0
|
||||
python-ceilometerclient>=1.5.0,<2.0
|
||||
python-cinderclient>=1.4.0,<2.0
|
||||
python-glanceclient>=1.1.0,<2.0
|
||||
python-heatclient>=0.8.0,<1.0
|
||||
python-novaclient>=2.30.1,<3.0
|
||||
python-openstackclient>=1.7.0,<2.0
|
||||
python-swiftclient>=2.6.0,<3.0
|
||||
pika>=0.10.0,<1.0
|
||||
distro-info
|
||||
# END: Amulet OpenStack Charm Helper Requirements
|
||||
|
|
113
tests/README
113
tests/README
|
@ -1,113 +0,0 @@
|
|||
This directory provides Amulet tests to verify basic deployment functionality
|
||||
from the perspective of this charm, its requirements and its features, as
|
||||
exercised in a subset of the full OpenStack deployment test bundle topology.
|
||||
|
||||
Reference: lp:openstack-charm-testing for full test bundles.
|
||||
|
||||
A single topology and configuration is defined and deployed, once for each of
|
||||
the defined Ubuntu:OpenStack release combos. The ongoing goal is for this
|
||||
charm to always possess tests and combo definitions for all currently-supported
|
||||
release combinations of U:OS.
|
||||
|
||||
test_* methods are called in lexical sort order, as with most runners. However,
|
||||
each individual test method should be idempotent and expected to pass regardless
|
||||
of run order or Ubuntu:OpenStack combo. When writing or modifying tests,
|
||||
ensure that every individual test is not dependent on another test_ method.
|
||||
|
||||
Test naming convention, purely for code organization purposes:
|
||||
1xx service and endpoint checks
|
||||
2xx relation checks
|
||||
3xx config checks
|
||||
4xx functional checks
|
||||
9xx restarts, config changes, actions and other final checks
|
||||
|
||||
In order to run tests, charm-tools and juju must be installed:
|
||||
sudo add-apt-repository ppa:juju/stable
|
||||
sudo apt-get update
|
||||
sudo apt-get install charm-tools juju juju-deployer amulet
|
||||
|
||||
Alternatively, tests may be exercised with proposed or development versions
|
||||
of juju and related tools:
|
||||
|
||||
# juju proposed version
|
||||
sudo add-apt-repository ppa:juju/proposed
|
||||
sudo apt-get update
|
||||
sudo apt-get install charm-tools juju juju-deployer
|
||||
|
||||
# juju development version
|
||||
sudo add-apt-repository ppa:juju/devel
|
||||
sudo apt-get update
|
||||
sudo apt-get install charm-tools juju juju-deployer
|
||||
|
||||
Some tests may need to download files. If a web proxy server is required in
|
||||
the environment, the AMULET_HTTP_PROXY environment variable must be set and
|
||||
passed into the juju test command. This is unrelated to juju's http proxy
|
||||
settings or behavior.
|
||||
|
||||
The following examples demonstrate different ways that tests can be executed.
|
||||
All examples are run from the charm's root directory.
|
||||
|
||||
* To run all +x tests in the tests directory:
|
||||
|
||||
bzr branch lp:charms/trusty/foo
|
||||
cd foo
|
||||
make functional_test
|
||||
|
||||
* To run the tests against a specific release combo as defined in tests/:
|
||||
|
||||
bzr branch lp:charms/trusty/foo
|
||||
cd foo
|
||||
juju test -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
|
||||
|
||||
* To run tests and keep the juju environment deployed after a failure:
|
||||
|
||||
bzr branch lp:charms/trusty/foo
|
||||
cd foo
|
||||
juju test --set-e -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
|
||||
|
||||
* To re-run a test module against an already deployed environment (one
|
||||
that was deployed by a previous call to 'juju test --set-e'):
|
||||
|
||||
./tests/015-basic-trusty-icehouse
|
||||
|
||||
* Even with --set-e, `juju test` will tear down the deployment when all
|
||||
tests pass. The following work flow may be more effective when
|
||||
iterating on test writing.
|
||||
|
||||
bzr branch lp:charms/trusty/foo
|
||||
cd foo
|
||||
./tests/setup/00-setup
|
||||
juju bootstrap
|
||||
./tests/015-basic-trusty-icehouse
|
||||
# make some changes, run tests again
|
||||
./tests/015-basic-trusty-icehouse
|
||||
# make some changes, run tests again
|
||||
./tests/015-basic-trusty-icehouse
|
||||
|
||||
* There may be test definitions in the tests/ dir which are not set +x
|
||||
executable. This is generally true for deprecated releases, or for
|
||||
upcoming releases which are not yet validated and enabled. To enable
|
||||
and run these tests:
|
||||
bzr branch lp:charms/trusty/foo
|
||||
cd foo
|
||||
ls tests
|
||||
chmod +x tests/017-basic-trusty-kilo
|
||||
./tests/setup/00-setup
|
||||
juju bootstrap
|
||||
./tests/017-basic-trusty-kilo
|
||||
|
||||
|
||||
Additional notes:
|
||||
|
||||
* Use DEBUG to turn on debug logging, use ERROR otherwise.
|
||||
u = OpenStackAmuletUtils(ERROR)
|
||||
u = OpenStackAmuletUtils(DEBUG)
|
||||
|
||||
* To interact with the deployed environment:
|
||||
export OS_USERNAME=admin
|
||||
export OS_PASSWORD=openstack
|
||||
export OS_TENANT_NAME=admin
|
||||
export OS_REGION_NAME=RegionOne
|
||||
export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -e trusty -f keystone`:5000/v2.0
|
||||
keystone user-list
|
||||
glance image-list
|
|
@ -0,0 +1,9 @@
|
|||
# Overview
|
||||
|
||||
This directory provides Amulet tests to verify basic deployment functionality
|
||||
from the perspective of this charm, its requirements and its features, as
|
||||
exercised in a subset of the full OpenStack deployment test bundle topology.
|
||||
|
||||
For full details on functional testing of OpenStack charms please refer to
|
||||
the [functional testing](http://docs.openstack.org/developer/charm-guide/testing.html#functional-testing)
|
||||
section of the OpenStack Charm Guide.
|
|
@ -106,26 +106,21 @@ class BasicDeployment(OpenStackAmuletDeployment):
|
|||
else:
|
||||
self.master_unit = self.find_master(ha=False)
|
||||
|
||||
for i in range(self.units):
|
||||
uid = 'percona-cluster/%d' % i
|
||||
unit = self.d.sentry.unit[uid]
|
||||
assert self.is_mysqld_running(unit), 'mysql not running: %s' % uid
|
||||
for unit in self.d.sentry['percona-cluster']:
|
||||
assert self.is_mysqld_running(unit), 'mysql not running: %s' % unit
|
||||
|
||||
def find_master(self, ha=True):
|
||||
for unit_id, unit in self.d.sentry.unit.items():
|
||||
if not unit_id.startswith('percona-cluster/'):
|
||||
continue
|
||||
|
||||
for unit in self.d.sentry['percona-cluster']:
|
||||
if not ha:
|
||||
return unit
|
||||
|
||||
# is the vip running here?
|
||||
output, code = unit.run('sudo ip a | grep "inet %s/"' % self.vip)
|
||||
print('---')
|
||||
print(unit_id)
|
||||
print(unit)
|
||||
print(output)
|
||||
if code == 0:
|
||||
print('vip(%s) running in %s' % (self.vip, unit_id))
|
||||
print('vip(%s) running in %s' % (self.vip, unit))
|
||||
return unit
|
||||
|
||||
def get_pcmkr_resources(self, unit=None):
|
||||
|
|
|
@ -0,0 +1,44 @@
|
|||
#!/usr/bin/env python
|
||||
# test percona-cluster (1 node)
|
||||
import basic_deployment
|
||||
|
||||
|
||||
class MultiNode(basic_deployment.BasicDeployment):
|
||||
def __init__(self):
|
||||
super(MultiNode, self).__init__(units=3, series='xenial')
|
||||
|
||||
def _get_configs(self):
|
||||
"""Configure all of the services."""
|
||||
cfg_percona = {'sst-password': 'ubuntu',
|
||||
'root-password': 't00r',
|
||||
'dataset-size': '512M',
|
||||
'vip': self.vip,
|
||||
'min-cluster-size': 3,
|
||||
'ha-bindiface': 'ens2'}
|
||||
|
||||
cfg_ha = {'debug': True,
|
||||
'corosync_mcastaddr': '226.94.1.4',
|
||||
'corosync_key': ('xZP7GDWV0e8Qs0GxWThXirNNYlScgi3sRTdZk/IXKD'
|
||||
'qkNFcwdCWfRQnqrHU/6mb6sz6OIoZzX2MtfMQIDcXu'
|
||||
'PqQyvKuv7YbRyGHmQwAWDUA4ed759VWAO39kHkfWp9'
|
||||
'y5RRk/wcHakTcWYMwm70upDGJEP00YT3xem3NQy27A'
|
||||
'C1w=')}
|
||||
|
||||
configs = {'percona-cluster': cfg_percona}
|
||||
if self.units > 1:
|
||||
configs['hacluster'] = cfg_ha
|
||||
|
||||
return configs
|
||||
|
||||
def run(self):
|
||||
super(MultiNode, self).run()
|
||||
msg = "Percona cluster failed to bootstrap"
|
||||
assert self.is_pxc_bootstrapped(), msg
|
||||
got = self.get_cluster_size()
|
||||
msg = "Percona cluster unexpected size (wanted=%s, got=%s)" % (3, got)
|
||||
assert got == '3', msg
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
t = MultiNode()
|
||||
t.run()
|
|
@ -11,8 +11,7 @@ class PauseResume(basic_deployment.BasicDeployment):
|
|||
|
||||
def run(self):
|
||||
super(PauseResume, self).run()
|
||||
uid = 'percona-cluster/0'
|
||||
unit = self.d.sentry.unit[uid]
|
||||
unit = self.d.sentry['percona-cluster'][0]
|
||||
assert self.is_mysqld_running(unit), 'mysql not running: %s' % uid
|
||||
assert utils.status_get(unit)[0] == "active"
|
||||
|
|
@ -20,7 +20,7 @@ class SingleNode(basic_deployment.BasicDeployment):
|
|||
def run(self):
|
||||
super(SingleNode, self).run()
|
||||
assert self.is_pxc_bootstrapped(), "Cluster not bootstrapped"
|
||||
sentry_unit = self.d.sentry.unit['percona-cluster/0']
|
||||
sentry_unit = self.d.sentry['percona-cluster'][0]
|
||||
|
||||
assert u.status_get(sentry_unit)[0] == "active"
|
||||
|
|
@ -1,18 +0,0 @@
|
|||
#!/bin/bash
|
||||
|
||||
set -ex
|
||||
|
||||
sudo add-apt-repository --yes ppa:juju/stable
|
||||
sudo apt-get update --yes
|
||||
sudo apt-get install --yes amulet \
|
||||
distro-info-data \
|
||||
python-cinderclient \
|
||||
python-distro-info \
|
||||
python3-distro-info \
|
||||
python-glanceclient \
|
||||
python-heatclient \
|
||||
python-keystoneclient \
|
||||
python-neutronclient \
|
||||
python-novaclient \
|
||||
python-pika \
|
||||
python-swiftclient
|
|
@ -1,21 +1,17 @@
|
|||
bootstrap: true
|
||||
reset: false
|
||||
virtualenv: true
|
||||
makefile:
|
||||
- lint
|
||||
- test
|
||||
sources:
|
||||
- ppa:juju/stable
|
||||
packages:
|
||||
- amulet
|
||||
- distro-info-data
|
||||
- python-ceilometerclient
|
||||
- python-cinderclient
|
||||
- python-distro-info
|
||||
- python-glanceclient
|
||||
- python-heatclient
|
||||
- python-keystoneclient
|
||||
- python-neutronclient
|
||||
- python-novaclient
|
||||
- python-pika
|
||||
- python-swiftclient
|
||||
# Bootstrap the model if necessary.
|
||||
bootstrap: True
|
||||
# Re-use bootstrap node instead of destroying/re-bootstrapping.
|
||||
reset: True
|
||||
# Use tox/requirements to drive the venv instead of bundletester's venv feature.
|
||||
virtualenv: False
|
||||
# Leave makefile empty, otherwise unit/lint tests will rerun ahead of amulet.
|
||||
makefile: []
|
||||
# Do not specify juju PPA sources. Juju is presumed to be pre-installed
|
||||
# and configured in all test runner environments.
|
||||
#sources:
|
||||
# Do not specify or rely on system packages.
|
||||
#packages:
|
||||
# Do not specify python packages here. Use test-requirements.txt
|
||||
# and tox instead. ie. The venv is constructed before bundletester
|
||||
# is invoked.
|
||||
#python-packages:
|
||||
|
|
37
tox.ini
37
tox.ini
|
@ -5,6 +5,8 @@ skipsdist = True
|
|||
[testenv]
|
||||
setenv = VIRTUAL_ENV={envdir}
|
||||
PYTHONHASHSEED=0
|
||||
AMULET_SETUP_TIMEOUT=2700
|
||||
passenv = AMULET_*
|
||||
install_command =
|
||||
pip install --allow-unverified python-apt {opts} {packages}
|
||||
commands = ostestr {posargs}
|
||||
|
@ -24,6 +26,41 @@ commands = flake8 {posargs} hooks unit_tests tests actions
|
|||
[testenv:venv]
|
||||
commands = {posargs}
|
||||
|
||||
[testenv:func27-noop]
|
||||
# DRY RUN - For Debug
|
||||
basepython = python2.7
|
||||
deps = -r{toxinidir}/requirements.txt
|
||||
-r{toxinidir}/test-requirements.txt
|
||||
commands =
|
||||
bundletester -vl DEBUG -r json -o func-results.json --test-pattern "gate-*" -n --no-destroy
|
||||
|
||||
[testenv:func27]
|
||||
# Charm Functional Test
|
||||
# Run all gate tests which are +x (expected to always pass)
|
||||
basepython = python2.7
|
||||
deps = -r{toxinidir}/requirements.txt
|
||||
-r{toxinidir}/test-requirements.txt
|
||||
commands =
|
||||
bundletester -vl DEBUG -r json -o func-results.json --test-pattern "gate-*" --no-destroy
|
||||
|
||||
[testenv:func27-smoke]
|
||||
# Charm Functional Test
|
||||
# Run a specific test as an Amulet smoke test (expected to always pass)
|
||||
basepython = python2.7
|
||||
deps = -r{toxinidir}/requirements.txt
|
||||
-r{toxinidir}/test-requirements.txt
|
||||
commands =
|
||||
bundletester -vl DEBUG -r json -o func-results.json gate-multiunit-xenial --no-destroy
|
||||
|
||||
[testenv:func27-dev]
|
||||
# Charm Functional Test
|
||||
# Run all development test targets which are +x (may not always pass!)
|
||||
basepython = python2.7
|
||||
deps = -r{toxinidir}/requirements.txt
|
||||
-r{toxinidir}/test-requirements.txt
|
||||
commands =
|
||||
bundletester -vl DEBUG -r json -o func-results.json --test-pattern "dev-*" --no-destroy
|
||||
|
||||
[flake8]
|
||||
ignore = E402,E226
|
||||
exclude = hooks/charmhelpers,tests/charmhelpers
|
||||
|
|
Loading…
Reference in New Issue