openstack/tempest
Code Issues Proposed changes

Add guidance on negative tests in HACKING.rst

Browse Source 
Adds guidance on negative tests based on work items from the Newton
design summit in Austin[1].

[1] https://etherpad.openstack.org/p/newton-qa-negative-testing

Change-Id: Ic44b0e17540a5e984006a4a74c8318ad7e74db31
This commit is contained in:
Chris Hoge 2016-06-23 16:03:28 -07:00
parent d23af14447
commit 2b47841c4d
1 changed files with 8 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
HACKING.rst
View File

@ -160,11 +160,17 @@ not there even if the cloud was configured with it.
Negative Tests
--------------
TODO: Write the guideline related to negative tests.
Error handling is an important aspect of API design and usage. Negative
tests are a way to ensure that an application can gracefully handle
invalid or unexpected input. However, as a black box integration test
suite, Tempest is not suitable for handling all negative test cases, as
the wide variety and complexity of negative tests can lead to long test
runs and knowledge of internal implementation details. The bulk of
negative testing should be handled with project function tests. The
exception to this rule is API tests used for interoperability testing.
Test skips because of Known Bugs
--------------------------------
If a test is broken because of a bug it is appropriate to skip the test until
bug has been fixed. You should use the skip_because decorator so that
Tempest's skip tracking tool can watch the bug status.