Update tests/README

This commit is contained in:
Ryan Beisner 2016-01-08 21:45:33 +00:00
parent aeab28fd7e
commit e22050545e

View File

@ -1,77 +1,113 @@
This directory provides Amulet tests that focus on verification of ceph
deployments.
This directory provides Amulet tests to verify basic deployment functionality
from the perspective of this charm, its requirements and its features, as
exercised in a subset of the full OpenStack deployment test bundle topology.
test_* methods are called in lexical sort order.
Reference: lp:openstack-charm-testing for full test bundles.
Test name convention to ensure desired test order:
A single topology and configuration is defined and deployed, once for each of
the defined Ubuntu:OpenStack release combos. The ongoing goal is for this
charm to always possess tests and combo definitions for all currently-supported
release combinations of U:OS.
test_* methods are called in lexical sort order, as with most runners. However,
each individual test method should be idempotent and expected to pass regardless
of run order or Ubuntu:OpenStack combo. When writing or modifying tests,
ensure that every individual test is not dependent on another test_ method.
Test naming convention, purely for code organization purposes:
1xx service and endpoint checks
2xx relation checks
3xx config checks
4xx functional checks
9xx restarts and other final checks
9xx restarts, config changes, actions and other final checks
Common uses of ceph relations in bundle deployments:
- [ nova-compute, ceph ]
- [ glance, ceph ]
- [ cinder, cinder-ceph ]
- [ cinder-ceph, ceph ]
More detailed relations of ceph service in a common deployment:
relations:
client:
- cinder-ceph
- glance
- nova-compute
mon:
- ceph
In order to run tests, you'll need charm-tools installed (in addition to
juju, of course):
In order to run tests, charm-tools and juju must be installed:
sudo add-apt-repository ppa:juju/stable
sudo apt-get update
sudo apt-get install charm-tools
sudo apt-get install charm-tools juju juju-deployer amulet
If you use a web proxy server to access the web, you'll need to set the
AMULET_HTTP_PROXY environment variable to the http URL of the proxy server.
Alternatively, tests may be exercised with proposed or development versions
of juju and related tools:
# juju proposed version
sudo add-apt-repository ppa:juju/proposed
sudo apt-get update
sudo apt-get install charm-tools juju juju-deployer
# juju development version
sudo add-apt-repository ppa:juju/devel
sudo apt-get update
sudo apt-get install charm-tools juju juju-deployer
Some tests may need to download files. If a web proxy server is required in
the environment, the AMULET_HTTP_PROXY environment variable must be set and
passed into the juju test command. This is unrelated to juju's http proxy
settings or behavior.
The following examples demonstrate different ways that tests can be executed.
All examples are run from the charm's root directory.
* To run all tests (starting with 00-setup):
* To run all +x tests in the tests directory:
make test
bzr branch lp:charms/trusty/foo
cd foo
make functional_test
* To run a specific test module (or modules):
* To run the tests against a specific release combo as defined in tests/:
juju test -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
bzr branch lp:charms/trusty/foo
cd foo
juju test -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
* To run a specific test module (or modules), and keep the environment
deployed after a failure:
* To run tests and keep the juju environment deployed after a failure:
juju test --set-e -v -p AMULET_HTTP_PROXY 15-basic-trusty-icehouse
bzr branch lp:charms/trusty/foo
cd foo
juju test --set-e -v -p AMULET_HTTP_PROXY 015-basic-trusty-icehouse
* To re-run a test module against an already deployed environment (one
that was deployed by a previous call to 'juju test --set-e'):
./tests/15-basic-trusty-icehouse
./tests/015-basic-trusty-icehouse
For debugging and test development purposes, all code should be idempotent.
In other words, the code should have the ability to be re-run without changing
the results beyond the initial run. This enables editing and re-running of a
test module against an already deployed environment, as described above.
* Even with --set-e, `juju test` will tear down the deployment when all
tests pass. The following work flow may be more effective when
iterating on test writing.
Manual debugging tips:
bzr branch lp:charms/trusty/foo
cd foo
./tests/setup/00-setup
juju bootstrap
./tests/015-basic-trusty-icehouse
# make some changes, run tests again
./tests/015-basic-trusty-icehouse
# make some changes, run tests again
./tests/015-basic-trusty-icehouse
* Set the following env vars before using the OpenStack CLI as admin:
export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0
export OS_TENANT_NAME=admin
* There may be test definitions in the tests/ dir which are not set +x
executable. This is generally true for deprecated releases, or for
upcoming releases which are not yet validated and enabled. To enable
and run these tests:
bzr branch lp:charms/trusty/foo
cd foo
ls tests
chmod +x tests/017-basic-trusty-kilo
./tests/setup/00-setup
juju bootstrap
./tests/017-basic-trusty-kilo
Additional notes:
* Use DEBUG to turn on debug logging, use ERROR otherwise.
u = OpenStackAmuletUtils(ERROR)
u = OpenStackAmuletUtils(DEBUG)
* To interact with the deployed environment:
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_TENANT_NAME=admin
export OS_REGION_NAME=RegionOne
* Set the following env vars before using the OpenStack CLI as demoUser:
export OS_AUTH_URL=http://`juju-deployer -f keystone 2>&1 | tail -n 1`:5000/v2.0
export OS_TENANT_NAME=demoTenant
export OS_USERNAME=demoUser
export OS_PASSWORD=password
export OS_REGION_NAME=RegionOne
export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju-deployer -e trusty -f keystone`:5000/v2.0
keystone user-list
glance image-list