openstack/tempest

History

Matt Riedemann 848805f21b Add test for compute API os-quota-class-sets This tests showing and updating the 'default' quota class, which is really all this API is used for. It was removed in Icehouse in part to not knowing it was used even though python-novaclient and Horizon were using it. The removal was reverted and this change is to enforce what the API is used for. The extension only lives in the Nova v2 API, it was removed in the v3 API in Icehouse. This change adds new quota client and test classes for the new API. An external lock fixture is used in all compute API quota tests to avoid introducing race bugs with the existing tenant-specific os-quota-sets tests. Closes-Bug: #1325727 Change-Id: Ib0cde08dfaa0f6a5e180d247864fb59d76eca903		2014-06-17 08:48:11 -07:00
..
baremetal	Use old dict generator syntax in baremetal/test_ports	2014-05-27 09:14:18 +02:00
compute	Add test for compute API os-quota-class-sets	2014-06-17 08:48:11 -07:00
data_processing	Merge "Sahara: add API tests for job binary internals"	2014-06-13 20:56:29 +00:00
database	Added Trove (database) version API tests	2014-05-05 13:34:46 -04:00
identity	Merge "tests the Keystone extensions V2 API"	2014-06-13 13:46:48 +00:00
image	Merge "Remove nova v2 client in ListSnapshotImagesTest"	2014-06-11 01:30:33 +00:00
network	Merge "Use a more sensible network topology for some router tests"	2014-06-14 06:25:40 +00:00
object_storage	Add bug skip for test_container_synchronization()	2014-06-09 15:31:59 +00:00
orchestration	Merge "Heat Overlapping ip issue"	2014-06-13 20:56:19 +00:00
queuing	Add Marconi Smoke Tests	2014-05-23 13:58:20 +00:00
telemetry	Add compute notifications tests for ceilometer	2014-06-03 13:50:19 +04:00
volume	Merge "adds the missing _interface variable"	2014-06-12 10:38:34 +00:00
__init__.py	Remove copyright from empty files	2014-01-14 03:02:04 +04:00
README.rst	Add 'Field' to the title of the Field Guides	2013-09-18 13:29:53 +09:00
utils.py	Prepare for enabling H302 (identity,volume,etc.)	2014-02-20 15:37:03 +09:00

README.rst

Tempest Field Guide to API tests

What are these tests?

One of Tempest's prime function is to ensure that your OpenStack cloud works with the OpenStack API as documented. The current largest portion of Tempest code is devoted to test cases that do exactly this.

It's also important to test not only the expected positive path on APIs, but also to provide them with invalid data to ensure they fail in expected and documented ways. Over the course of the OpenStack project Tempest has discovered many fundamental bugs by doing just this.

In order for some APIs to return meaningful results, there must be enough data in the system. This means these tests might start by spinning up a server, image, etc, then operating on it.

Why are these tests in tempest?

This is one of the core missions for the Tempest project, and where it started. Many people use this bit of function in Tempest to ensure their clouds haven't broken the OpenStack API.

It could be argued that some of the negative testing could be done back in the projects themselves, and we might evolve there over time, but currently in the OpenStack gate this is a fundamentally important place to keep things.

Scope of these tests

API tests should always use the Tempest implementation of the OpenStack API, as we want to ensure that bugs aren't hidden by the official clients.

They should test specific API calls, and can build up complex state if it's needed for the API call to be meaningful.

They should send not only good data, but bad data at the API and look for error codes.

They should all be able to be run on their own, not depending on the state created by a previous test.