Add unit test framework nose and associated helper scripts

For usage documentation, refer to heat/tests/testing-overview.txt.
run_tests.sh is what runs the tests.

Fixes #44

Signed-off-by: Jeff Peeler <jpeeler@redhat.com>
This commit is contained in:
Jeff Peeler 2012-04-13 17:10:30 -04:00
parent b070322351
commit 2f8e95baec
13 changed files with 524 additions and 47 deletions

View File

@ -0,0 +1,16 @@
--- .nova-venv/lib/python2.6/site-packages/eventlet/green/subprocess.py.orig
2011-05-25
23:31:34.597271402 +0000
+++ .nova-venv/lib/python2.6/site-packages/eventlet/green/subprocess.py
2011-05-25
23:33:24.055602468 +0000
@@ -32,7 +32,7 @@
setattr(self, attr, wrapped_pipe)
__init__.__doc__ = subprocess_orig.Popen.__init__.__doc__
- def wait(self, check_interval=0.01):
+ def wait(self, check_interval=0.01, timeout=None):
# Instead of a blocking OS call, this version of wait() uses logic
# borrowed from the eventlet 0.2 processes.Process.wait() method.
try:

View File

@ -0,0 +1,5 @@
def setup():
print "package setup complete"
def teardown():
print "package teardown complete"

View File

@ -0,0 +1,2 @@
type
area

View File

@ -0,0 +1,32 @@
###
### an unparented test -- no encapsulating class, just any fn starting with
### 'test'.
### http://darcs.idyll.org/~t/projects/nose-demo/simple/tests/test_stuff.py.html
###
import sys
import nose
from nose.plugins.attrib import attr
from nose import with_setup
# module level
def setUp():
print "test1 setup complete"
def tearDown():
print "test1 teardown complete"
@with_setup(setUp, tearDown) # test level
@attr(tag=['example', 'func'])
@attr(speed='fast')
def test_a():
assert 'a' == 'a'
print "assert a"
def test_b():
assert 'b' == 'b'
print "assert b"
if __name__ == '__main__':
sys.argv.append(__file__)
nose.main()

View File

@ -0,0 +1,26 @@
###
### non-unittest derived test -- class is instantiated, then functions
### starting with 'test' are executed.
### http://darcs.idyll.org/~t/projects/nose-demo/simple/tests/test_stuff.py.html
###
import sys
import nose
from nose.plugins.attrib import attr
# sets attribute on all test methods
@attr(tag=['example', 'class'])
@attr(speed='fast')
class TestClass:
def test2(self):
assert 'b' == 'b'
print "assert b"
def setUp(self):
print "test2 setup complete"
def tearDown(self):
print "test2 teardown complete"
if __name__ == '__main__':
sys.argv.append(__file__)
nose.main()

View File

@ -0,0 +1,25 @@
###
### the standard unittest-derived test
### http://darcs.idyll.org/~t/projects/nose-demo/simple/tests/test_stuff.py.html
###
import sys
import nose
import unittest
from nose.plugins.attrib import attr
# sets attribute on all test methods
@attr(tag=['example', 'unittest'])
@attr(speed='fast')
class ExampleTest(unittest.TestCase):
def test_a(self):
self.assert_(1 == 1)
def setUp(self):
print "test3 setup complete"
def tearDown(self):
print "test3 teardown complete"
if __name__ == '__main__':
sys.argv.append(__file__)
nose.main()

View File

@ -0,0 +1,122 @@
Heat testing
------------
All tests are to be placed in the heat/tests directory. The directory is
organized by test type (unit, functional, etc). Within each type
directory one may create another directory for additional test files as
well as a separate __init__.py, which allows setup and teardown code to
be shared with the tests present in the same directory.
An example directory structure illustrating the above:
heat/tests
|-- examples
| |-- __init__.py <-- tests1-3 will execute the fixtures (setup and
| |-- test1.py teardown routines) only once
| |-- test2.py
| |-- test3.py
|-- __init__.py
`-- unit
|-- __init__.py
|-- test_template_convert.py
If a given test has no overlapping requirements (variables or same
routines) a new test does not need to create a subdirectory under the
test type.
Implementing a test
-------------------
Nose, the testing framework - http://pypi.python.org/pypi/nose, finds on
demand available tests to run. The name of the file must contain "test"
or "Test" at a word boundary. The recommended format is for the test to
be named test_<NAME>.
There are many different ways to write a test. Three different ways are
present in the tests/examples directory. The differences are slight
enough to just describe the make up of just one test.
---
Example 1:
import sys
import nose
from nose.plugins.attrib import attr
from nose import with_setup
# module level
def setUp():
print "test1 setup complete"
def tearDown():
print "test1 teardown complete"
@with_setup(setUp, tearDown) # test level
@attr(tag=['example', 'func'])
def test_a():
assert 'a' == 'a'
print "assert a"
def test_b():
assert 'b' == 'b'
print "assert b"
# allows testing of the test directly, shown below
if __name__ == '__main__':
sys.argv.append(__file__)
nose.main()
---
Example 1 illustrates fixture execution at the test, module, and package
level:
$ python test1.py -s
package setup complete
test1 setup complete
test1 setup complete
assert a
test1 teardown complete
.assert b
.test1 teardown complete
package teardown complete
----------------------------------------------------------------------
Ran 2 tests in 0.001s
OK
All fixtures are optional. In the above output you can trace the order
execution of the fixtures relative to the tests. Fixtures at the class
level are present in example 2, which consists of simply defining them
within the class.
Note the attribute decorator with a list of values, which functionality
is provided via the attributeselector plugin. This "tag" allows running
tests all matching the assigned attribute(s). Tests should always
include the tag attribute with at least these values: <test type>,
<test area>. Also an attribute of speed should be used with a value of
either slow, normal, or fast. Following this convention allows for finer
granular testing without having to find the specific tests to run.
Running the tests
-----------------
There is a run_tests.sh script in the top level of the tree. The script
will by default execute all found tests, but can be modified with the
tag argument:
$ ./run_tests.sh -V -a tag=example # (runs all the examples)
There are two important options provided by the run_tests.sh script that
should have special attention. The '--virtual-env' or '-V' will build
and run all the tests inside of an isolated python environment located
in the .venv directory. It's sort of like mock just for python :)
The other option of note is the '--pep8' or '-p' flag. This is a python
style checker that is good to run periodically. Pep8 is automatically
executed when tests are run inside the virtual environment since pep8 is
intentionally installed.
Please see ./run_tests.sh -h for future enhancements and/or minor
non-documented functionality.

View File

@ -1,41 +0,0 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2011 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import json
import unittest
from heat.engine.json2capexml import *
class ParseTestCase(unittest.TestCase):
def setUp(self):
pass
def tearDown(self):
pass
def test_01(self):
done=False
with open('templates/WordPress_Single_Instance.template') as f:
blob = json.load(f)
cape_transformer = Json2CapeXml(blob, 'WordPress_Single_Instance')
cape_transformer.convert()
print cape_transformer.get_xml()
done=True
self.assertTrue(done)

View File

@ -7,7 +7,7 @@ function usage {
echo " -V, --virtual-env Always use virtualenv. Install automatically if not present"
echo " -N, --no-virtual-env Don't use virtualenv. Run tests in local environment"
echo " -f, --force Force a clean re-build of the virtual environment. Useful when dependencies have been added."
echo " --unittests-only Run unit tests only, exclude functional tests."
echo " --unittests-only Run unit tests only."
echo " -p, --pep8 Just run pep8"
echo " -h, --help Print this usage message"
echo ""
@ -19,11 +19,12 @@ function usage {
function process_option {
case "$1" in
-h|--help) usage;;
-V|--virtual-env) let always_venv=1; let never_venv=0;;
-N|--no-virtual-env) let always_venv=0; let never_venv=1;;
-f|--force) let force=1;;
--unittests-only) noseargs="$noseargs --exclude-dir=heat/tests/functional";;
--unittests-only) noseargs="$noseargs -a tag=unit";;
-p|--pep8) let just_pep8=1;;
-h|--help) usage;;
*) noseargs="$noseargs $1"
esac
}
@ -41,13 +42,19 @@ for arg in "$@"; do
process_option $arg
done
NOSETESTS="python run_tests.py $noseargs"
function run_tests {
# Just run the test suites in current environment
${wrapper} rm -f tests.sqlite
${wrapper} $NOSETESTS 2> run_tests.err.log
}
NOSETESTS="python run_tests.py $noseargs"
function run_pep8 {
echo "Running pep8 ..."
PEP8_OPTIONS="--exclude=$PEP8_EXCLUDE --repeat"
PEP8_INCLUDE="bin/*.py heat tools setup.py run_tests.py"
${wrapper} pep8 $PEP8_OPTIONS $PEP8_INCLUDE
}
if [ $never_venv -eq 0 ]
then
@ -69,11 +76,20 @@ then
if [ "x$use_ve" = "xY" -o "x$use_ve" = "x" -o "x$use_ve" = "xy" ]; then
# Install the virtualenv and run the test suite in it
python tools/install_venv.py
wrapper=${with_venv}
wrapper=${with_venv}
fi
fi
fi
fi
if [ $just_pep8 -eq 1 ]; then
run_pep8
exit
fi
run_tests || exit
if [ -z "$noseargs" ]; then
run_pep8
fi

246
tools/install_venv.py Normal file
View File

@ -0,0 +1,246 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Copyright 2010 OpenStack, LLC
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""
Installation script for Heat's development virtualenv
"""
import optparse
import os
import subprocess
import sys
ROOT = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
VENV = os.path.join(ROOT, '.venv')
PIP_REQUIRES = os.path.join(ROOT, 'tools', 'pip-requires')
TEST_REQUIRES = os.path.join(ROOT, 'tools', 'test-requires')
PY_VERSION = "python%s.%s" % (sys.version_info[0], sys.version_info[1])
def die(message, *args):
print >> sys.stderr, message % args
sys.exit(1)
def check_python_version():
if sys.version_info < (2, 6):
die("Need Python Version >= 2.6")
def run_command_with_code(cmd, redirect_output=True, check_exit_code=True):
"""
Runs a command in an out-of-process shell, returning the
output of that command. Working directory is ROOT.
"""
if redirect_output:
stdout = subprocess.PIPE
else:
stdout = None
proc = subprocess.Popen(cmd, cwd=ROOT, stdout=stdout)
output = proc.communicate()[0]
if check_exit_code and proc.returncode != 0:
die('Command "%s" failed.\n%s', ' '.join(cmd), output)
return (output, proc.returncode)
def run_command(cmd, redirect_output=True, check_exit_code=True):
return run_command_with_code(cmd, redirect_output, check_exit_code)[0]
class Distro(object):
def check_cmd(self, cmd):
return bool(run_command(['which', cmd], check_exit_code=False).strip())
def install_virtualenv(self):
if self.check_cmd('virtualenv'):
return
if self.check_cmd('easy_install'):
print 'Installing virtualenv via easy_install...',
if run_command(['easy_install', 'virtualenv']):
print 'Succeeded'
return
else:
print 'Failed'
die('ERROR: virtualenv not found.\n\nHeat development'
' requires virtualenv, please install it using your'
' favorite package management tool')
def post_process(self):
"""Any distribution-specific post-processing gets done here.
In particular, this is useful for applying patches to code inside
the venv."""
pass
class Fedora(Distro):
"""This covers all Fedora-based distributions.
Includes: Fedora, RHEL, CentOS, Scientific Linux"""
def check_pkg(self, pkg):
return run_command_with_code(['rpm', '-q', pkg],
check_exit_code=False)[1] == 0
def yum_install(self, pkg, **kwargs):
print "Attempting to install '%s' via yum" % pkg
run_command(['sudo', 'yum', 'install', '-y', pkg], **kwargs)
def apply_patch(self, originalfile, patchfile):
run_command(['patch', originalfile, patchfile])
def install_virtualenv(self):
if self.check_cmd('virtualenv'):
return
if not self.check_pkg('python-virtualenv'):
self.yum_install('python-virtualenv', check_exit_code=False)
super(Fedora, self).install_virtualenv()
def post_process(self):
"""Workaround for a bug in eventlet.
This currently affects RHEL6.1, but the fix can safely be
applied to all RHEL and Fedora distributions.
This can be removed when the fix is applied upstream
Nova: https://bugs.launchpad.net/nova/+bug/884915
Upstream: https://bitbucket.org/which_linden/eventlet/issue/89"""
# Install "patch" program if it's not there
if not self.check_pkg('patch'):
self.yum_install('patch')
# Apply the eventlet patch
self.apply_patch(os.path.join(VENV, 'lib', PY_VERSION, 'site-packages',
'eventlet/green/subprocess.py'),
'contrib/redhat-eventlet.patch')
def get_distro():
if os.path.exists('/etc/fedora-release') or \
os.path.exists('/etc/redhat-release'):
return Fedora()
else:
return Distro()
def check_dependencies():
get_distro().install_virtualenv()
def create_virtualenv(venv=VENV, no_site_packages=True):
"""Creates the virtual environment and installs PIP only into the
virtual environment
"""
print 'Creating venv...',
if no_site_packages:
run_command(['virtualenv', '-q', '--no-site-packages', VENV])
else:
run_command(['virtualenv', '-q', VENV])
print 'done.'
print 'Installing pip in virtualenv...',
if not run_command(['tools/with_venv.sh', 'easy_install',
'pip>1.0']).strip():
die("Failed to install pip.")
print 'done.'
def pip_install(*args):
run_command(['tools/with_venv.sh',
'pip', 'install', '--upgrade'] + list(args),
redirect_output=False)
def install_dependencies(venv=VENV):
print 'Installing dependencies with pip (this can take a while)...'
# First things first, make sure our venv has the latest pip and distribute.
pip_install('pip')
pip_install('distribute')
# Install greenlet by hand - just listing it in the requires file does not
# get it in stalled in the right order
pip_install('greenlet')
pip_install('-r', PIP_REQUIRES)
pip_install('-r', TEST_REQUIRES)
# Tell the virtual env how to "import nova"
#pthfile = os.path.join(venv, "lib", PY_VERSION, "site-packages",
# "nova.pth")
#f = open(pthfile, 'w')
#f.write("%s\n" % ROOT)
def post_process():
get_distro().post_process()
def print_help():
help = """
Heat development environment setup is complete.
Heat development uses virtualenv to track and manage Python dependencies
while in development and testing.
To activate the Heat virtualenv for the extent of your current shell
session you can run:
$ source .venv/bin/activate
Or, if you prefer, you can run commands in the virtualenv on a case by case
basis by running:
$ tools/with_venv.sh <your command>
Also, make test will automatically use the virtualenv.
"""
print help
def parse_args():
"""Parse command-line arguments"""
parser = optparse.OptionParser()
parser.add_option("-n", "--no-site-packages", dest="no_site_packages",
default=False, action="store_true",
help="Do not inherit packages from global Python install")
return parser.parse_args()
def main(argv):
(options, args) = parse_args()
check_python_version()
check_dependencies()
create_virtualenv(no_site_packages=options.no_site_packages)
install_dependencies()
post_process()
print_help()
if __name__ == '__main__':
main(sys.argv)

16
tools/pip-requires Normal file
View File

@ -0,0 +1,16 @@
# The greenlet package must be compiled with gcc and needs
# the Python.h headers. Make sure you install the python-dev
# package to get the right headers...
greenlet>=0.3.1
eventlet>=0.9.12
PasteDeploy
routes
webob==1.0.8
argparse
swift
sqlalchemy-migrate>=0.7
httplib2
kombu
iso8601>=0.1.4
python-novaclient

8
tools/test-requires Normal file
View File

@ -0,0 +1,8 @@
# Packages needed for dev testing
distribute>=0.6.24
coverage
nose
nosexcover
openstack.nose_plugin
pep8==0.6.1

4
tools/with_venv.sh Executable file
View File

@ -0,0 +1,4 @@
#!/bin/bash
TOOLS=`dirname $0`
VENV=$TOOLS/../.venv
source $VENV/bin/activate && $@