From a5afb4fc3e7fb8d253eb1d8b08ef1884641f7759 Mon Sep 17 00:00:00 2001 From: Marc Koderer Date: Tue, 11 Feb 2014 15:38:15 +0100 Subject: [PATCH] Adapt documentation for negative testing With the introduction of the new negative testing framework the workflow for new test has changed. Partially-implements: bp negative-tests Change-Id: I522c2be227a120a6a714349fa29b31e60b5e0bf4 --- HACKING.rst | 36 ++++++++++++++++++++++++++++-------- 1 file changed, 28 insertions(+), 8 deletions(-) diff --git a/HACKING.rst b/HACKING.rst index 3fa1ff5bb4..1566b2f1ed 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -107,16 +107,36 @@ they do not need to be tagged as compute. Negative Tests -------------- -When adding negative tests to tempest there are 2 requirements. First the tests -must be marked with a negative attribute. For example:: +Newly added negative tests should use the negative test framework. First step +is to create an interface description in a json file under `etc/schemas`. +These descriptions consists of two important sections for the test +(one of those is mandatory): - @attr(type=negative) - def test_resource_no_uuid(self): - ... + - A resource (part of the URL of the request): Resources needed for a test + must be created in `setUpClass` and registered with `set_resource` e.g.: + `cls.set_resource("server", server['id'])` -The second requirement is that all negative tests must be added to a negative -test file. If such a file doesn't exist for the particular resource being -tested a new test file should be added. + - A json schema: defines properties for a request. + +After that a test class must be added to automatically generate test scenarios +out of the given interface description: + + class SampeTestNegativeTestJSON(, test.NegativeAutoTest): + _interface = 'json' + _service = 'compute' + _schema_file = 'compute/servers/get_console_output.json' + scenarios = test.NegativeAutoTest.generate_scenario(_schema_file) + +Negative tests must be marked with a negative attribute:: + + @test.attr(type=['negative', 'gate']) + def test_get_console_output(self): + self.execute(self._schema_file) + +All negative tests should be added into a separate negative test file. +If such a file doesn't exist for the particular resource being tested a new +test file should be added. Old XML based negative tests can be kept but should +be renamed to `_xml.py`. Test skips because of Known Bugs --------------------------------