History

Matthew Treinish c795b9e136 Enable E113,E111,H302,F812,E713 rules After bumping the hacking version to the 0.9.x series ignores were added for several rules. This commit fixes the violations for a subset of these rules and re-enables the checks. Change-Id: I8cc92a5de5acf43a2143751851cf6fddab401ba4		2014-06-17 23:33:30 +00:00
..
simple_read_only	Enable E113,E111,H302,F812,E713 rules	2014-06-17 23:33:30 +00:00
README.rst	Add 'Field' to the title of the Field Guides	2013-09-18 13:29:53 +09:00
__init__.py	Merge "don't log cli output on success"	2014-03-24 20:02:51 +00:00
output_parser.py	Add a new exception for invalid structure	2014-03-25 11:40:58 +09:00

README.rst

Tempest Field Guide to CLI tests

What are these tests?

The cli tests test the various OpenStack command line interface tools to ensure that they minimally function. The current scope is read only operations on a cloud that are hard to test via unit tests.

Why are these tests in tempest?

These tests exist here because it is extremely difficult to build a functional enough environment in the python-*client unit tests to provide this kind of testing. Because we already put up a cloud in the gate with devstack + tempest it was decided it was better to have these as a side tree in tempest instead of another QA effort which would split review time.

Scope of these tests

This should stay limited to the scope of testing the cli. Functional testing of the cloud should be elsewhere, this is about exercising the cli code.

Example of a good test

Tests should be isolated to a single command in one of the python clients.

Tests should not modify the cloud.

If a test is validating the cli for bad data, it should do it with assertRaises.

A reasonable example of an existing test is as follows:

def test_admin_list(self):
    self.nova('list')
    self.nova('list', params='--all-tenants 1')
    self.nova('list', params='--all-tenants 0')
    self.assertRaises(subprocess.CalledProcessError,
                      self.nova,
                      'list',
                      params='--all-tenants bad')