Tempest Coding Guide

* Adding HACKING.rst as Tempest Coding Guide
* Fixing the section/title formater markup's length

Change-Id: Ic61d287cd3bbf3adc5f7d98b17957a5be125f613
This commit is contained in:
Attila Fazekas 2013-06-09 16:35:23 +02:00
parent 7a3e004acc
commit 23fdf1de42
8 changed files with 31 additions and 26 deletions

View File

@ -1,3 +1,7 @@
Tempest Coding Guide
====================
Test Data/Configuration Test Data/Configuration
----------------------- -----------------------
- Assume nothing about existing test data - Assume nothing about existing test data

1
doc/source/HACKING.rst Symbolic link
View File

@ -0,0 +1 @@
../../HACKING.rst

View File

@ -3,9 +3,9 @@
You can adapt this file completely to your liking, but it should at least You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive. contain the root `toctree` directive.
=================================== =======================
Tempest Testing Project Tempest Testing Project
=================================== =======================
Contents: Contents:
@ -13,11 +13,11 @@ Contents:
:maxdepth: 2 :maxdepth: 2
overview overview
HACKING
------------
-------------------------------
Field Guides Field Guides
------------------------------- ------------
Tempest contains tests of many different types, the field guides Tempest contains tests of many different types, the field guides
attempt to explain these in a way that makes it easy to understand attempt to explain these in a way that makes it easy to understand
where your test contributions should go. where your test contributions should go.

View File

@ -1,9 +1,9 @@
Tempest Guide to API tests Tempest Guide to API tests
======== ==========================
What are these tests? What are these tests?
-------- ---------------------
One of Tempest's prime function is to ensure that your OpenStack cloud One of Tempest's prime function is to ensure that your OpenStack cloud
works with the OpenStack API as documented. The current largest works with the OpenStack API as documented. The current largest
@ -21,7 +21,7 @@ spinning up a server, image, etc, then opperating on it.
Why are these tests in tempest? Why are these tests in tempest?
-------- -------------------------------
This is one of the core missions for the Tempest project, and where it This is one of the core missions for the Tempest project, and where it
started. Many people use this bit of function in Tempest to ensure started. Many people use this bit of function in Tempest to ensure
@ -34,7 +34,7 @@ place to keep things.
Scope of these tests Scope of these tests
-------- --------------------
API tests should always use the Tempest implementation of the API tests should always use the Tempest implementation of the
OpenStack API, as we want to ensure that bugs aren't hidden by the OpenStack API, as we want to ensure that bugs aren't hidden by the

View File

@ -1,16 +1,16 @@
Tempest Guide to CLI tests Tempest Guide to CLI tests
======== ==========================
What are these tests? What are these tests?
--------- ---------------------
The cli tests test the various OpenStack command line interface tools The cli tests test the various OpenStack command line interface tools
to ensure that they minimally function. The current scope is read only to ensure that they minimally function. The current scope is read only
operations on a cloud that are hard to test via unit tests. operations on a cloud that are hard to test via unit tests.
Why are these tests in tempest? Why are these tests in tempest?
--------- -------------------------------
These tests exist here because it is extremely difficult to build a These tests exist here because it is extremely difficult to build a
functional enough environment in the python-*client unit tests to functional enough environment in the python-*client unit tests to
provide this kind of testing. Because we already put up a cloud in the provide this kind of testing. Because we already put up a cloud in the
@ -20,14 +20,14 @@ would split review time.
Scope of these tests Scope of these tests
--------- --------------------
This should stay limited to the scope of testing the cli. Functional This should stay limited to the scope of testing the cli. Functional
testing of the cloud should be elsewhere, this is about exercising the testing of the cloud should be elsewhere, this is about exercising the
cli code. cli code.
Example of a good test Example of a good test
--------- ----------------------
Tests should be isolated to a single command in one of the python Tests should be isolated to a single command in one of the python
clients. clients.

View File

@ -1,9 +1,9 @@
Tempest Guide to Scenario tests Tempest Guide to Scenario tests
======== ===============================
What are these tests? What are these tests?
-------- ---------------------
Scenario tests are "through path" tests of OpenStack Scenario tests are "through path" tests of OpenStack
function. Complicated setups where one part might depend on completion function. Complicated setups where one part might depend on completion
@ -17,13 +17,13 @@ from that snapshot.
Why are these tests in tempest? Why are these tests in tempest?
-------- -------------------------------
This is one of tempests core purposes, testing the integration between This is one of tempests core purposes, testing the integration between
projects. projects.
Scope of these tests Scope of these tests
-------- --------------------
Scenario tests should always test at least 2 services in Scenario tests should always test at least 2 services in
interaction. They should use the official python client libraries for interaction. They should use the official python client libraries for
OpenStack, as they provide a more realistic approach in how people OpenStack, as they provide a more realistic approach in how people
@ -34,7 +34,7 @@ services they exercise.
Example of a good test Example of a good test
-------- ----------------------
While we are looking for interaction of 2 or more services, be While we are looking for interaction of 2 or more services, be
specific in your interactions. A giant "this is my data center" smoke specific in your interactions. A giant "this is my data center" smoke
test is hard to debug when it goes wrong. test is hard to debug when it goes wrong.

View File

@ -1,5 +1,5 @@
Tempest Field Guide to Stress Tests Tempest Field Guide to Stress Tests
====================================================== ===================================
Nova is a distributed, asynchronous system that is prone to race condition Nova is a distributed, asynchronous system that is prone to race condition
bugs. These bugs will not be easily found during bugs. These bugs will not be easily found during
@ -9,7 +9,7 @@ in a more controlled environment.
Environment Environment
------------ -----------
This particular framework assumes your working Nova cluster understands Nova This particular framework assumes your working Nova cluster understands Nova
API 2.0. The stress tests can read the logs from the cluster. To enable this API 2.0. The stress tests can read the logs from the cluster. To enable this
you have to provide the hostname to call 'nova-manage' and you have to provide the hostname to call 'nova-manage' and

View File

@ -1,9 +1,9 @@
Tempest Guide to Whitebox tests Tempest Guide to Whitebox tests
======== ===============================
What are these tests? What are these tests?
-------- ---------------------
When you hit the OpenStack API, this causes internal state changes in When you hit the OpenStack API, this causes internal state changes in
the system. This might be database transitions, vm modifications, the system. This might be database transitions, vm modifications,
@ -20,7 +20,7 @@ be useful for validating Tempest internals.
Why are these tests in tempest? Why are these tests in tempest?
-------- -------------------------------
Especially when it comes to something like VM state changing, which is Especially when it comes to something like VM state changing, which is
a coordination of numerous running daemons, and a functioning VM, it's a coordination of numerous running daemons, and a functioning VM, it's
@ -28,7 +28,7 @@ very difficult to get a realistic test like this in unit tests.
Scope of these tests Scope of these tests
-------- --------------------
White box tests should be limitted to tests where black box testing White box tests should be limitted to tests where black box testing
(using the OpenStack API to verify results) isn't sufficient. (using the OpenStack API to verify results) isn't sufficient.
@ -40,7 +40,7 @@ internals changes in OpenStack projects.
Example of a good test Example of a good test
-------- ----------------------
Pushing VMs through a series of state transitions, and ensuring along Pushing VMs through a series of state transitions, and ensuring along
the way the database state transitions match what's expected. the way the database state transitions match what's expected.