This commit moves the oslo config generator config file from living as
a data file, which makes it very difficult to rely on, to just living
in the package. By including it in the package we have a guaranteed
relative path we can rely on for building utilities on it, like the
tempest init command.
Change-Id: I5b9deae733f42fe3ed39418103304f71b43dccb3
Partially-implements: bp tempest-run-cmd
This commit simplifies the venv management via tox for tempest jobs.
All tox targets that actually run tempest will now share a single tox
venv .tox/tempest. This also adds a new target tempest-venv which is
the same as the pre-existing venv target, but uses the common venv.
We can't simply use the common venv with the venv target because
external tooling, like the docs jobs, rely on the .tox/venv path
existing when using the venv target.
Depends-On: I34a69020eee07156e64026781a3c0bffdb5ab415
Change-Id: Iab2e6e04b6c5795a4d0c8214564106525b942308
This commit adds tox jobs to run tempest with ostestr instead of the
pretty_tox*.sh wrappers. ostestr provides essentially the same
functionality except it exposes slightly more rich selection logic
and additional debugging options. However, since the cli isn't
backwards compatible with pretty_tox we can't simply remove the old
tox jobs.
Change-Id: I347438b1d561af7e05fdce895e2e3e51d2b38b0c
This commit aligns multiple lines in tox for visual consistency.
I fixed them to 4 white spaces because the tox example[1] shows 4 white
spaces indentation.
[1] http://tox.readthedocs.org/en/latest/example/basic.html
Change-Id: I4f3b9c5c4982a0e96bb120ce8546dd48b0fa8a4b
This isn't really a scenario test, it's a performance test, however
without any of the infrastructure to do performance testing /
comparisons. It's time has come to say goodbye.
Depends-On: I788cd70dc4e6a9c3f8e6805d5bdcdf7f549fce35
Which removes it's use from the gate.
Change-Id: I93b2fb33e97381f7c1e0cb1ef09ebc5c42c16ecc
Tempest is going to be on pypi and expose a stable interface in
the tempest.lib namespace, so adding reno to it.
Partially-implements: bp tempest-lib-reintegration
Change-Id: Ice4c27f7bc6d0154cfced9789f3c82236106a9e3
The idempotent id decorator and the check-uuid script were migrated to
tempest-lib as part of tempest-lib 0.10.0. However, tempest was never
updated to use the library version of idempotent id or check-uuid.
This commit updates tempest to remove it's local copies of the
idempotent_id decorator and the check-uuid tool and use them from
tempest-lib instead.
Also, instead of updating the import of every single test module in
tempest an alias is just added to tempest.test to let all the old
imports work seamlessly. Updating the imports and removing the alias
may or may not be worth it in the future, but that will be a follow
on patch or discussion.
Change-Id: Iae3f491ada8f7b55c1fd817e033d968c2102d11e
To solve the problem of "db type could not be determined" on py34
we have to run first the py34 env to, then, run py27. This patch
puts py34 first in the tox.ini list of envs to avoid this problem
to happen.
Closes-bug: #1489059
Change-Id: Ia11ecf7bd147e35b7c8ad3db0eaad17f893e78ba
There is a lot of H404/405 violations in Tempest now, and that leads
difficult to migrate the code to tempest-lib or the other projects'
repos. This patch fixes these violations for remaining.
Change-Id: I3abac40ebb33836980c119d59bc97f035b213c0a
Init command was not working properly as it could not locate the
configuration sample file.
Steps taken:
- Move config-generator.tempest.conf to etc so it will be installed
with pbr.
- Update all files with new path to config-generator-tempest.conf as
needed.
- Refactor init command so if it detects we are not in a virtual
environment, try to find the global config directory /etc/tempest.
If that fails fallback to [sys.prefix]/etc/tempest.
Closes-Bug: #1491058
Closes-Bug: #1490670
Change-Id: I960bc711ff78ac2b0441ef63dff8ec4fb268fd3a
Earlier we specified, that we must generate sample config
file before generating docs. Now after commit 13ed69fff
got merged, we must update the comments or any info related
to generating config file.
Change-Id: Ic0686322f07045700ff390eea8faa05eebe3df7c
The expectation from users is that to build docs locally all you need
to do is run build_sphinx on the repo and the docs will be fully
created. When we added the sample config to the tempest docs we ignored
this and just added the sample config generation to the docs tox job.
This commit fixes this by baking in the sample config generation to
the sphinx conf.py file which should ensure we have the sample config
generated every time that we call build_sphinx.
Change-Id: Ifd570707847f52592f5bc9bc743b8d3c98512639
This removes the check on in tree sample generation to ensure the file
matches the config options. However, to counter the loss of the in-tree
the tempest init command will now automatically generate a new sample
config in the local etc dir when run. This will include all of the oslo
options that were previously included in the in-tree sample file.
Additionally, the docs now contain the complete text of the generated
sample output. These also get published on docs.openstack.org on each
commit to tempest. Between this 2 alternatives there shouldn't be a need
to keep the file in tree anymore, especially given the weekly headaches
with new oslo releases changing the config option help breaking the job.
Change-Id: Ie3bc368329d240ca9604cc2f9b2cac544b3ee715
Depends-On: I4064a041a965ed2419b68efc8dc31fce37b24cfd
Since including the sample configuration in tree is
becoming difficult to maintain, lets try moving it to
the developer docs. We still have the ability to view it
easily for users, and it can be auto-generated on doc build,
so it should also always be up to date.
Change-Id: I7ec2d76448feef5ecb9af7c238bbd12619fa31d8
If site-packages is deactivated tempest plugins aren't visible.
Create a new tox entry point instead of activating it globally.
Change-Id: If6cd204d4c1e64cf7bcf79bd5d32d3c10a1b2559
Closes-bug: #1478559
When we run tempest on the machine behind proxy,the pip command failed to
connect to pypi.python.org to download packages.
Fix it by add another option "passenv" to [testenv] section.
Change-Id: Iedc40a2266be59a2298d5909486a1439f2757b8c
Closes-Bug: #1463978
Tox 2.0 introduced env variable separation, meaning that env vars set
in the env calling tox will not automatically be in the test env. This
breaks some assumptions we have in tempest. This commit adds all the
env vars which are used by testr or tempest directly for running tests
in the hope of making things work on tox 2.0 and older versions.
Change-Id: Icfe4bb5df0e48cbc9fcdbc602656672295d1887e
This commit adds references explaining the current state of python 3.4
and tempest to the readme, pkg metadata, and adds an etry to the tox
envlist for running the unit tests on python 3.4.
Change-Id: I9c1e3cdf53dbae723a9fee67ad96932709d2305f
The cli tests have been marked for removal for ~6 months and the
framework was the first thing included in tempest lib. There has
been more than enough time for all the projects to pick this up
in the client repos. So let's remove it all!
As part of this a couple of missing entries for tempest's
requirements.txt were found. These dependencies were being installed
by the clients so the fact they were missing was never noticed prior
to this. This commit also adds these missing entries back into the
requirements file.
Change-Id: I4f8638f1c048bbdb598dd181f4af272ef9923806
The decision to include all of scenario by default makes for an
extremely heavy test of tests in smoke, we should instead only include
specific scenario tests that expose some basic sanity
checking. Advanced scenario tests should be dropped from this list.
This change means we no longer have to exclude 'slow' tests, as we'll
just tag the right tests with smoke.
Change-Id: I34aa8aa9274e02bb68f62863d64c47ba9916f2e4
Adds uuid4 as a decorator of the form:
@test.idempotent_id('12345678-1234-1234-1234-123456789abc')
to every test in the Tempest tree. Includes a gate check to
ensure the existence and uniqueness of the ids.
Modify check tool to ignore Tempest unit tests.
Change-Id: I19e3c7dd555a3ea09d585fb9091c357a300e6559
Co-Authored-By: Sergey Slipushenko <sslypushenko@mirantis.com>
Implements: bp test-uuid
This patch adds a new tool to check for existence and uniqueness
of UUIDs across Tempest or any other test repository based on
Tempest. The tool also includes an option to automatically tag
a test repository with UUIDs if they don't exist. The tool
will be used in the gate to ensure UUID existence.
Change-Id: I25aa83c7836f5a607af2aaa4bf862fa72766f799
Co-Authored-By: Sergey Slipushenko <sslypushenko@mirantis.com>
Partially-Implements: bp test-uuid
Fully decouple the versions needed for tempest from the underlying
system. This will help make it possible to pin library versions on
stable branches to the library version used when the branch was
cut, while allowing master global requirements to move beyond that
version.
Note: We previously enabled sitepackages everywhere to avoid
recompiling indirect dependencies that were pulled in via the
pythonclients. To address this we were working on removing the CLI tests
from tempest.
Change-Id: I3b355fb592dbbd38cb6805ec413e02644088b088
* tox.ini: If setuptools is declared as a dependency, even
transitively, it can trigger a race in some versions of pip when
called with -U wherein pip removes setuptools in preparation for
upgrading it, and then proceeds to install something else using
setuptools. Work around this behavior by upgrading setuptools within
the virtualenv in advance.
Change-Id: I6d9bf61c1ce41e46ff6f621bfa88d511f6c16120
Related-Bug: #1405579
It's extremely easy for incorrect local results to happen if a file
has been moved or deleted in an upstream pull. Add the rm pyc cantrip
to ensure local testing matches what a clean upstream is going to be
like as much as possible.
Change-Id: Iecf18d7c2310918ec90c14c64cd3083e46459980
The default for tempest in .testr.conf is 500s. This was too short for
the heat slow tests so the time for that was increased to 1200. This
change is to a periodic job so if we need to it can be increased more
in the future.
Change-Id: If27ddef9865c54b997130c707382d2014d82e0a6
Tempest has moved to new oslo-config-generator and this bug
workaround is no longer needed.
Change-Id: I50ffdcbc8a8f4830ba9c274af4c908e96467dad5
Closes-Bug: 1365136
Also fixes tempest reported bug with old generator in oslo-incubator,
when identical options are registered in different groups.
Change-Id: I274f48503c1b61ca09cabc8d8c50245ba1059943
Closes-bug: 1282887
This commit creates a separate base env for tempest tests and
tempest's unit tests. However, to keep the same interface the tempest
env is setup by overloading the testenv using substitution from a
common base section.
As part of this internal reorganization in the tox.ini several other
things are cleaned up or removed. Usless or duplicate job definitions
are removed. The unit tests are switched to pretty_tox. Previously
they were run without any output which made it difficult to figure out
exactly what was running. Additionally, this removes the unnecessary
regex filter on the unit test job. The unit tests use a different
discovery path so there is no need to also filter it on the same path
this will allow the use of more specific regexes with tox to filter
tests.
Change-Id: I996d81dec2739bbe907f5fe87a335994c01697b3
Stop triggering triggering a “not installed in virtualenv” every time
bash is called. Since we use this in all tempest based jobs, we have
these unnecessary warnings all over the place.
Change-Id: I8af53c0241dbcd656c274944338384be0f2f250d
We've ironed out remaining issues that crop up when running
with a random seed. This updates tox.ini to revert to the default
behavior of running with PYTHONHASHSEED=random. The pep8 check
continues to run with PYTHONHASHSEED=0 to workaround oslo issues
around config value ordering in the check_uptodate.sh test.
Change-Id: Ifa0dba8c3970fc676d374f33c129ef0f9d526c94
To run the tempest unit tests with coverage enabled the coverage
module needs to be installed. This requirement is properly listed in
test-requirements.txt, however the coverage job definition wasn't
installing coverage. This commit fixes this oversight by adding the
test-requirements file as a dependency for the job.
Change-Id: Ic443c6c4a3166e6b198cbe2af0a77ff2f0a38e83
This removes the setenv override and sets the required
PYTHONHASHSEED=0 for the 'all' tox target, which devstack-vm-gate.sh
ends up calling for the current Ironic job.
Change-Id: Icc6a1cae1e49140c4e4648f6c75ad415f92479ce
Closes-bug: #1359929