openstack
/
python-novaclient
Code Issues Proposed changes
python-novaclient/novaclient/tests/functional
History
Sean Dague 420dc2884a refactor functional test base class to no inherit from tempest_lib 
Test base classes inheriting outside the existing source tree to
anything higher up the stack than testtools is really an anti
pattern. It makes it *far* less clear what is going on. We really need
to own and understand our base class setup for tests.

This does that unwind, so that we only now call out to tempest_lib in
specific ways (like building clients, using decorators). The timeout
and log capture pieces are pulled inline.

At the end of this we end up with a base test that defines:

 - self.client - a Nova API client
 - self.flavor - a workable flavor for booting guests
 - self.image - a workable image for booting guests
 - self.cli_clients - tempest_lib cli clients

Change-Id: I716be51d7d1825a757934298f06b2f04d64cf0dd
 		2015-04-21 11:06:00 -04:00
..
hooks Add first pass at post_test_hook for functional tests 2015-02-04 16:32:18 -08:00
README.rst Add functional testing README 2015-03-09 14:41:02 -07:00
__init__.py First pass at tempest_lib based functional testing 2015-01-28 15:44:43 -08:00
base.py refactor functional test base class to no inherit from tempest_lib 2015-04-21 11:06:00 -04:00
test_instances.py refactor functional test base class to no inherit from tempest_lib 2015-04-21 11:06:00 -04:00
test_readonly_nova.py Copy functional tests from tempest cli read only 2015-02-11 14:55:26 -08:00
test_volumes_api.py refactor functional test base class to no inherit from tempest_lib 2015-04-21 11:06:00 -04:00

README.rst

python-novaclient functional testing

Idea

Over time we have noticed two issues with novaclient unit tests.

  • Does not exercise the CLI
  • We can get the expected server behavior wrong, and test the wrong thing.

We are using functional tests, run against a running cloud (primarily devstack), to address these two cases.

Additionally these functional tests can be considered example uses of python-novaclient.

These tests started out in tempest as read only nova CLI tests, to make sure the CLI didn't simply stacktrace when being used (which happened on multiple occasions).

Testing Theory

We are treating python-novaclient as legacy code, so we do not want to spend a lot of effort adding in missing features. In the future the CLI will move to python-openstackclient, and the python API will be based on the OpenStack SDK project. But until that happens we still need better functional testing, to prevent regressions etc.

Since python-novaclient has two uses, CLI and python API, we should have two sets of functional tests. CLI and python API. The python API tests should never use the CLI. But the CLI tests can use the python API where adding native support to the CLI for the required functionality would involve a non trivial amount of work.

Functional Test Guidelines

  • Consume credentials via standard client environmental variables:

    OS_USERNAME
OS_PASSWORD
OS_TENANT_NAME
OS_AUTH_URL

  • Try not to require an additional configuration file