openstack
/
tempest
Code Issues Proposed changes
tempest/tempest/api
History
Marc Koderer b2978da5ab Deduplicate negative test calls 
Instead of duplicating the execute call for each negative test cases,
defining a generic way for all tests by registering a test function
dynamically on basis of the test class itself.

This is done using a class decorator (not inheritance) since it's
not possible to get the child class name on import time.

Change-Id: Ife0a8ea95ecfcac8a5f9c89babd74df0d5f884fe
Partially-implements: bp autogen-negative-tests
 		2014-03-26 13:56:09 +01:00
..
baremetal Convert ironic, swift, and heat api tests to use global CONF object 2014-01-30 19:56:51 +00:00
compute Deduplicate negative test calls 2014-03-26 13:56:09 +01:00
data_processing Rename Savanna to Sahara 2014-03-13 23:59:22 +04:00
database Add Trove (database) Flavor API Tests 2014-03-05 21:46:45 -08:00
identity Merge "Some keystone V3 API tests throw incorrect errors" 2014-03-26 07:32:19 +00:00
image Stop leaking test_update_images 2014-03-12 20:12:08 +09:00
network Merge "Refactor create_ and update_ methods for floating ips" 2014-03-25 17:59:36 +00:00
object_storage Object storage tests to use default auth_provider 2014-03-13 10:35:18 +00:00
orchestration simplify heat test_limits 2014-03-25 17:05:10 -04:00
queuing Add basic Delete Queue Marconi test 2014-03-12 12:14:19 -04:00
telemetry Adds more verification in list alarms 2014-03-25 06:57:58 +00:00
volume Merge "Stop volume leaking" 2014-03-24 12:52:10 +00:00
README.rst Add 'Field' to the title of the Field Guides 2013-09-18 13:29:53 +09:00
__init__.py Remove copyright from empty files 2014-01-14 03:02:04 +04:00
utils.py Prepare for enabling H302 (identity,volume,etc.) 2014-02-20 15:37:03 +09:00

README.rst

Tempest Field Guide to API tests

What are these tests?

One of Tempest's prime function is to ensure that your OpenStack cloud works with the OpenStack API as documented. The current largest portion of Tempest code is devoted to test cases that do exactly this.

It's also important to test not only the expected positive path on APIs, but also to provide them with invalid data to ensure they fail in expected and documented ways. Over the course of the OpenStack project Tempest has discovered many fundamental bugs by doing just this.

In order for some APIs to return meaningful results, there must be enough data in the system. This means these tests might start by spinning up a server, image, etc, then operating on it.

Why are these tests in tempest?

This is one of the core missions for the Tempest project, and where it started. Many people use this bit of function in Tempest to ensure their clouds haven't broken the OpenStack API.

It could be argued that some of the negative testing could be done back in the projects themselves, and we might evolve there over time, but currently in the OpenStack gate this is a fundamentally important place to keep things.

Scope of these tests

API tests should always use the Tempest implementation of the OpenStack API, as we want to ensure that bugs aren't hidden by the official clients.

They should test specific API calls, and can build up complex state if it's needed for the API call to be meaningful.

They should send not only good data, but bad data at the API and look for error codes.

They should all be able to be run on their own, not depending on the state created by a previous test.