When installing tempest, it could be that the root of tempest code
is not a subfolder of the folder where the testrepository
configuration is held, and where test results are stored.
unittest2 will refuse to run tests for which the discovery root
is not a subfolder of the specified top level folder.
To fully support this scenario, adding OS_TOP_LEVEL in .testr.conf
and defaulting it to "./" which is the current value.
Change-Id: I60f788051d6a10ab00aecef5d09093a68f7f4030
This commit adds a new env variable OS_TEST_LOCK_PATH to the
.testr.conf file to set the lock path for using oslo locks.
Previously, if an external lock couldn't be used to isolate test cases
because the unit tests do not have a config file. By getting a lock
path with an env variable and setting the location in the config
fixture external locks can be used by first invoking the config
fixture.
Partially implements bp unit-tests
Change-Id: I08c27ab125183f5d055eaa45c4f0e0a527445475
As of commit 87f772c0bd the default
value for OS_TEST_PATH as set in .testr.conf is no longer valid.
In addition, because shell variables are evaluated in a
somewhat unintuitive order in situations like this, the default
wasn't being used properly anyway. See link below for an
explanation why:
http://stackoverflow.com/questions/10938483/bash-specifying-environment-variables-for-echo-on-command-line
This commit fixes both problems.
Change-Id: I8eec146a21eab656cb48b9e23a92a9da52246269
This commit adds the env variable OS_TEST_PATH to specify the test
discovery path for testr. This can be used to specify which path will
be used for test discovery.
Closes-Bug: #1260710
Change-Id: Idcc8f6b5d82a2adc1a1f8ad04a47e8d694dc7de7
This commit increases the default timeout for a test specified in
.testr.conf from 250 secs to 500 secs. The default build timeout
used for waiting for resource deletion is 400 secs. By using a test
timeout that was smaller than the build timeout some slow async
resource creation and deletions weren't completing before the test
was marked as failed for taking to long
Change-Id: Ia6292e68a8c3a0a0c03667a15f528c2ec6a5fbed
Partial-Bug: #1218585
This commit adds environmental variables to .testr.conf to disable
testr capturing STDOUT and STDERR by default. So now STDOUT and
STDERR will print immediately by default when using testr. It also
adds a default 250 sec timeout for tests. If a test takes longer than
this it will fail.
Change-Id: I32d8b1a856f6e7932d16b1464553e9c25fbc3cfc
This commit removes the unnecessary class filter run_test_classes.py
and reverts the .testr.conf to the standard format. The class filter
is not needed when using test grouping.
Change-Id: I460c48121c26d07eb29f355c07a2059f5e6b24b0
This commit switches the test runner in tempest to testr from nose for all
jobs that aren't gating. This will allow the usage of parallel testing with
the use of a group_regex in testr. Group_regex will ensure that the classes
get scheduled together and the run times are tracked together. Than the
tools/run_test_classes.py script will filter the test_ids and pass only the
classes to subunit.run to ensure we are only running setupClass once.
This commit also adds a new option to run_tests.sh. -t/--with-testr can be
used to optionally run tempest in parallel with testr. Once running with
testr gets stable enough this will become the default for run_tests.
It also adds a testr-full tox job so we can have a tracking non-voting job
that runs the same tests as the gate but with testr.
Implements: blueprint speed-up-tempest
Change-Id: Iedc9bd92b8f8471c60c614c7d7c05046d7b32743
Adds the testrepository config file needed to use testr to run the
tempest tests. To run the tests just do:
testr run
or to run them in parallel:
testr run --parallel
At this stage you may encounter one or two failures with --parallel due to
conflicts between a couple of tests that not have yet been fixed, but a serial
run should work fine.
Change-Id: I08121fdcd48e2f4ff86bf9e1d4d378f818699d89