Heat testing
------------

All tests are to be placed in the heat/tests directory. The directory is
organized by test type (unit, functional, etc). Within each type
directory one may create another directory for additional test files as
well as a separate __init__.py, which allows setup and teardown code to
be shared with the tests present in the same directory.

An example directory structure illustrating the above:

heat/tests
|-- examples
|   |-- __init__.py <-- tests1-3 will execute the fixtures (setup and
|   |-- test1.py        teardown routines) only once
|   |-- test2.py
|   |-- test3.py
|-- __init__.py
`-- unit
    |-- __init__.py
    |-- test_template_convert.py

If a given test has no overlapping requirements (variables or same
routines) a new test does not need to create a subdirectory under the
test type.

Implementing a test
-------------------

Nose, the testing framework - http://pypi.python.org/pypi/nose, finds on
demand available tests to run. The name of the file must contain "test"
or "Test" at a word boundary. The recommended format is for the test to
be named test_<NAME>.

There are many different ways to write a test. Three different ways are
present in the tests/examples directory. The differences are slight
enough to just describe the make up of just one test.

---
Example 1:

import sys
import nose
from nose.plugins.attrib import attr
from nose import with_setup

# module level
def setUp():
    print "test1 setup complete"

def tearDown():
    print "test1 teardown complete"

@with_setup(setUp, tearDown) # test level
@attr(tag=['example', 'func'])
def test_a():
    assert 'a' == 'a'
    print "assert a"

def test_b():
    assert 'b' == 'b'
    print "assert b"

# allows testing of the test directly, shown below
if __name__ == '__main__':
    sys.argv.append(__file__)
    nose.main()
---

Example 1 illustrates fixture execution at the test, module, and package
level:

$ python test1.py -s
package setup complete
test1 setup complete
test1 setup complete
assert a
test1 teardown complete
.assert b
.test1 teardown complete
package teardown complete

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK


All fixtures are optional. In the above output you can trace the order
execution of the fixtures relative to the tests. Fixtures at the class
level are present in example 2, which consists of simply defining them
within the class. 

Note the attribute decorator with a list of values, which functionality
is provided via the attributeselector plugin. This "tag" allows running
tests all matching the assigned attribute(s). Tests should always
include the tag attribute with at least these values: <test type>,
<test area>. Also an attribute of speed should be used with a value of
either slow, normal, or fast. Following this convention allows for finer
granular testing without having to find the specific tests to run.

If new dependencies are introduced upon the development of a test, the
tools/test-requires file needs to be updated so that the virtual
environment will be able to successfully execute all tests.

Running the tests
-----------------

There is a run_tests.sh script in the top level of the tree. The script
will by default execute all found tests, but can be modified with the
tag argument:

$ ./run_tests.sh -V -a tag=example  # (runs all the examples)

There are two important options provided by the run_tests.sh script that
should have special attention. The '--virtual-env' or '-V' will build
and run all the tests inside of an isolated python environment located
in the .venv directory. It's sort of like mock just for python :)

The other option of note is the '--pep8' or '-p' flag. This is a python
style checker that is good to run periodically. Pep8 is automatically
executed when tests are run inside the virtual environment since pep8 is
intentionally installed.

Please see ./run_tests.sh -h for future enhancements and/or minor
non-documented functionality.