Commit Graph

68 Commits (7416b91cf74776530eafc62cc9b83c49a8103748)

Author SHA1 Message Date
Ghanshyam Mann 25c441c76a Fix python classifier for py39
Temepest test and support python 3.9 so
fixing then python classifier in setup.cfg

Change-Id: Ida4ca6e04f6485c5c11e95ff708558267a827d2a
6 months ago
Ghanshyam Mann 24a3a3667a Migrate tempest jobs to Ubuntu Jammy (22.04)
As per 2023.1 cycle testing runtime and community goal[1] we need to migrate
upstream CI/CD to Ubuntu Jammy(22.04). Most of the Tempest jobs will be migrate
automatically once devstack base job start running on Jammy(Depends-On).
As we need to run at least one job on Ubuntu Focal(20.04), I am adding
a new job tempest-full-ubuntu-focal.

Few job like tempest-full-multinode-py3 and plugin sanity job needs to override
the nodeset. multinode job needs to run on Focal till stable/zed and on Jammy
from master(2023.1) onwards. Stable branch jobs testing stable branch needs to
keep running on their supported distro version which is Focal.



Change-Id: I2e37603b098880b778100613800496b01167b22f
6 months ago
Ghanshyam Mann 87661fcef7 Drop py3.6 and py3.7 from Tempest
In zed cycle, OpenStack projects and oslo lib
has dropped the py3.6 and py3.7 support


With oslo not supporting the py3.6 and py3.7, Tempest master is
failing and cannot be run on py3.6|7 env.


We discussed about how to solve it in ML[1] and QA meeting[2] and
decided to drop it from Tempest too. Stable branches testing from
stable/victoria to stable/yoga (currently supported branch by Tempest
master) can be tested on py3.8 or if cloud run those releases on py3.6
then Tempest can be run in virtual env with py38.

Closes-Bug: #1975036


Change-Id: Ie844887a798a006794729e733748ceaa5e06d2db
1 year ago
Ghanshyam Mann 1aac05c7b2 Test all supported python version for stable branch
As Tempest is branchless and needs to support older
supported stable branches, let's test it with all the
python version supported in those supported stable branches.
For that, we need to explicitly dd the py jobs instead of using
the template. This way, we will explicitly know that we are testing
all supported python versions when the new release template bumps the
min python version in the template.

In Yoga testing, we are making py3.9 job to run as voting, so
adding py3.9 job as voting.

Change-Id: I422da8da06509e529f4091e21cdc3d392d3cf910
2 years ago
Stephen Finucane a298fdf7a5 setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead


Change-Id: I532be460dc673548be9ed3b540e9feb8ca832fbe
Signed-off-by: Stephen Finucane <>
2 years ago
Ghanshyam Mann de69845d75 Switch testing to Victoria testing runtime
Updating the testing template from ussuri to victoria
testing runtime

Change-Id: If964909a73a52836e3adbb7b695f1c7322b81830
3 years ago
Zuul 1e844a898e Merge "Remove the deprecated tempest-account-generator CLI" 3 years ago
Andreas Jaeger b8435b27ac Remove universal wheel building
- Remove obsolete sections from setup.cfg:
  * Wheel is not needed for python 3 only repo

Change-Id: I95a717b873c8c21eeec4617b66c42cd6dbf226b2
3 years ago
Ghanshyam Mann 7cd0186dbe Replace old requires-python with python-requires
requires-python is old value and does not work, python-requires
is right metadata.

'python-requires' is introduced from PEP440 and requires-python was old value
in PEP345.

Change-Id: Id4a5197ec5dcaab419304e9244f4c6d319e8b5c6
3 years ago
Soniya Vyas ddcd4f44ac Remove the deprecated tempest-account-generator CLI
Tempest CLI tempest-account-generator was deprecated
4 years ago.
devstack still uses the deprecated CLI for which
patch has been proposed.

As deprecation is done long time back,
'tempest-account-generator' should be now used as
'tempest account-generator'

The following commit gives the idea of deprecation

Depends-on: I624e1dc57a3d3533322fb298c01f70241d0400ed
Signed-off by: Soniya Vyas<>
Change-Id: I7c18998dd877d7fd065e4d1d0950b6f6413587c3
3 years ago
Zuul c6747367f4 Merge "Remove the deprecated verify-tempest-config CLI" 3 years ago
Ghanshyam Mann 551e8d7a59 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

Tempest is ready with python 3 and ok to drop the
python 2.7 support. Last Tempest version officially supported
for py2.7 is 23.0.0.

Now onwards, Tempest will test all stable branch with py3 jobs.
tempest-full and other py2 version of jobs definition are kept
for stable branch gate.

Complete discussion & schedule can be found in

Ussuri Communtiy-wide goal:

Change-Id: I75868d5c9b6630fe78958ff89e58a0aced09a6b3
3 years ago
Ghanshyam b0a8057fbb Drop python 3.5 support from Tempest
Tempest, its plugins and their dependencies in ussuri
cycle are python-requires>=py3.6 which makes distro not
having python 3.6 to do hack to install py3.6 etc.

We cannot do >py3.6 in basepython so one way is to drop the
py3.5 support and basepython to py3 so that it can choose
the available py version in env.

It time to drop the py3.5 from Tempest. Last supported version of
Tempest for py3.5 is 23.0.0.

oslo drop py2 so we need to remove the py2 unit test job which
is failing and blocking gate.

There is seoarate work to drop the py2 things from Tempest in
compplete set.

Change-Id: I872dbb57df46d1193c0d409bb00d56f372d295fe
Related-Bug: 1861308
3 years ago
Soniya Vyas 4acf175de7 Remove the deprecated verify-tempest-config CLI
Tempest CLI verify-tempest-config was deprecated
4 years ago.

With assumption of all the users of this CLI have
moved to new CLI 'tempest verify_config', it is
time to remove the old deprecated CLI now.
'verify_tempest_config' should be now used as
'tempest verify_config'

The following commit gives the idea of deprecation

Signed-off by: Soniya Vyas<>
Change-Id: Ic3dbd6f5e854e017a5bec4b1a625730ab68e767d
3 years ago
Soniya Vyas 55ad7cd2bf Added tempest subcommand for subunit_describe_calls
We are adding a subcommand to tempest that is a parser for subunit
streams to determine what REST API calls are made inside of a test
and in what order they are called. Right now the implementation of
utility is as a console script and which is in argparse.

We are porting it to cliff to give a similar experience across
the codebase.

Change-Id: If95042898d8b2a148246c991d73d1a577fbcf7a0
Signed-off-by: Soniya Vyas <>
4 years ago
Corey Bryant 7409c72cb3 Add Python 3 Train unit tests
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.

See the Train python3-updates goal document for details:

Note: This diverges from the Train goal as it does not drop the
openstack-python35-jobs. As tempest is branchless it needs to continue
supporting Python 3.5.

Change-Id: Ie702ca637afb8229b6e605abcec25550bc9a0673
Story: #2005924
Task: #34240
4 years ago
zhangbailin cf4fda3838 Change openstack-dev to openstack-discuss
Mailinglists have been updated. Openstack-discuss replaces

Change-Id: I52361048bb1b10b7c26503bf47ad0b6b21e9eb64
5 years ago
junboli 263bfd833d Follow the new PTI for document build
This review already follows the new PTI

However, we also should remove [build_sphinx]
section as described in:

Change-Id: Ifbfe80024731d2c552a5aab90e025f9ebd593c4d
5 years ago
Ihar Hrachyshka 8032b2e154 Disable warnings as errors for sphinx
If GENERATE_TEMPEST_PLUGIN_LIST=False, the plugin-registry page is not
generated, and then sphinx fails with:

doc/source/index.rst:69:toctree contains reference to nonexisting
document u'plugin-registry'

Setting the variable to False is a legit use case, used in isolated
build environments like CBS used for RDO packages and should be

Alternatively, we could make inclusion of the page conditional to the
environment variable value, for example using 'only' directive for
sphinx and passing an appropriate 'tag' as a flag indicating the page is
generated; interface doesn't seem to provide an explicit way to
pass tags but SPHINXOPTS variable would be handy. Sadly, table of
contents can't be controlled with the directive:

Another alternative would be to generate a stub page with no useful
info just to upease sphinx. I figured it's easier to tweak sphinx
configuration though.

The fix still leaves warnings-as-errors enabled for tox environments
with assumption that if tox is used, the build environment is not as

Change-Id: Id174d1679c7b141b0327d8c23d6b14f7db83fae8
5 years ago
Jenkins 6843807419 Merge "Remove support for py34" 6 years ago
howardlee 27127378b6 Remove support for py34
Due to the change from Ubuntu Trusty to Xenial, only python3.5 is
available(Python3.4 is restricted to <= Mitaka). So, there is no
need to continue to keep these settings.

Change-Id: I07bf6c3299a2e509a2d30342922459dc79c8f281
6 years ago
chenxing e98720a22f Update the documention for doc migration
Change-Id: Ie6d322d1b9c93118af7c74986993010b9f4b76b2
6 years ago
Masayuki Igawa 00effc933f Use cliff.lister for tempest workspace command
This commit makes tempest workspace command use cliff.lister to show
various formats. And this commit splits the original class -
TempestWorkspace into the subcommand classes to use the cliff.lister
class. With this splitting, we can get the subcommand helps as a side

Change-Id: I07c69c467743a2f132c99d992773a53bda3bec7b
6 years ago
Stephen Finucane d114804f1a Use Sphinx 1.5 warning-is-error
With pbr 2.0 and Sphinx 1.5, the setting for treat sphinx warnings as
errors is setting warning-is-error in build_sphinx section. Migrate
the setting from the old warnerrors one.

There are a couple of small items to be cleaned up.

Change-Id: I01b1ac49c69d001ea25a475d903311d0b592656e
6 years ago
Jordan Pittier 2e7ae7c6e5 Remove the Stress framework
It's not used anymore. There was general consensus in Feb 2016
to deprecate it (see [1]) and remove it in Newton.

[1]  [qa] deprecating Tempest stress framework

Change-Id: Ib229985ea2a1fee495c9492c9ce1781e6bac1dc6
7 years ago
Luigi Toscano cdc0c172f7 tempest-cleanup can only be invoked as tempest cleanup
The CLI entry point removed here points to the main() class which was removed
so that tempest-cleanup is not a standalone command anymore.

See I2119acd402b0985d4a81c85501ce7265f782d37b

Change-Id: I3421667916bf3652ab0e777fa78c382a3f6e849d
7 years ago
Jenkins 0e8859b2fe Merge "Add python 3.5 classifier and venv" 7 years ago
ghanshyam 2bff1a133a
Remove deprecated Javelin CLI utilities from Tempest
Javelin CLI utilities has been deprecated in Jan 2016
- Ie4ab5258946271a16e96a070abeedfafa5423714
which is over 6 months of Tempest deprecation period.

This patch removes the javelin CLI and corresponding doc/sample
files etc.

Change-Id: Ia59d3e81f245f29a7bf0f56b690586dfc55f16a0
7 years ago
guo yunxian c78bda0031 Add python 3.5 classifier and venv
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.

Change-Id: Iaec0c9e88cb38ab876b5bc957f53989615cab246
7 years ago
Jenkins 396593432c Merge "Add subunit-describe-calls" 7 years ago
Stephen Lowrie c8548fc93c Add subunit-describe-calls
Adds new command subunit-describe-calls, documentation,
and unittests.

subunit-describe-calls is a parser for subunit v1 & v2 streams to
determine what REST API calls are made inside of a test and in what
order they are called.

Later commits will add additional functionality relating to request
& response headers & body data along with a stdout output option and
stdin input.

Change-Id: I468d0d3e3b6098da95a81cc86d9bdd1b47ee1f03
7 years ago
Matthew Treinish a051c22ad0
Add tempest run command
This commit adds a new run command to the unified cli endpoint. The
intent here is for tempest to control it's own run story. This
implements the basic runner and selection functionality to use the
command, however it's not necessarily the end state of the command.
The functionality in this patch is just a starting point to add the
command and the basic functionality needed. It is starting with a
limited feature set with the intent to add additional, more complex
functionality in self contained patches after the command exists.

Co-Authored-by: David Paterson <>
Co-Authored-by: Stephen Lowrie <>

Partially-Implements bp tempest-run-cmd

Depends-On: I09299043e536521d48dbe10632621138e3a366e0
Change-Id: I24588b5c00d005320e8719cf82b5dd95662572cf
7 years ago
step6829 80c14eca47 Centralized Workspaces
Create a consistent means for creation and management of Tempest

Creates a file located at ~/.tempest/workspaces.yaml which stores existing

Available subcommands: list, register, rename, move, remove

bp centralized-workspaces

Change-Id: I9595e3ba809e457951a0ffdf4b15f641f2fec4f4
7 years ago
Jenkins bb3f88ffbc Merge "Update the home page" 7 years ago
Andrea Frittoli (andreaf) db9672e347 Use tempest.lib code in tempest
Now that tempest_lib code is copied back into tempest, stop using
tempest_lib in tempest, and start using the copied code.

Remove the dependency to tempest_lib from requirements, and drop
the script to use tempest_lib in tempest. Add os-testr to the

Partially implements bp tempest-lib-reintegration

Change-Id: I21ab5fe6349f72c98ac9f960a29bf62e813f8b1b
7 years ago
jinxingfang ed45625ff0 Update the home page --->

Change-Id: Ia520ad7ce08d4b8dae168140440cad2928647470
7 years ago
David Paterson 68b8b9d26f Migrate tempest-account-generator to new cliff cli framework
Leave existing endpoint for now as to not disrupt existing
workflows and deprecate next cycle.

Add deprication warning for old usage.

Change-Id: I9db27a853184126a7f751800bc8cacd7dd4e40fd
Implements: blueprint tempest-cli-improvements
7 years ago
David Paterson e45aa842bc Migrate verify-tempest-config to new cliff cli framework
Leave existing endpoint for now as to not disrupt existing
workflows and deprecate next cycle.

Added deprecation message for old entry point.

Change-Id: If8afab441f51e5cf0f3bf1e2b95d76b2c1d0c474
Implements: blueprint tempest-cli-improvements
7 years ago
step6829 a9a664cde9 Add tempest-list-plugins
Add the tempest-list-plugins cmd.

bp list-plugins

Change-Id: If28311bc2e8d29a97ee46d7d73edba2a93aed7ce
7 years ago
Masayuki Igawa e5c7028236 Add run_stress to cliff-based cli framework
Also disable ability to run script in stand-alone mode. So users need to
use 'tempest run-stress' instead.

Change-Id: I3effd1b71b2375f75a11f5924205741be0903361
Implements: blueprint tempest-cli-improvements
8 years ago
David Paterson 07661de3ae Add cleanup to cliff-based cli framework
Also disable ability to run script in stand-alone mode so
users will use 'tempest cleanup' instead.

Change-Id: I2119acd402b0985d4a81c85501ce7265f782d37b
Implements: blueprint tempest-cli-improvements
8 years ago
Matthew Treinish f45ba2e604
Cleanup tempest docs a bit
This commit fixes a few things in the tempest docs. First it fixes all
of the sphinx warnings and enables fail on warn to ensure we're using
valid sphinx everywhere. It also adds a link from the configuration
guide to the sample config file.

Change-Id: I3ad645a6bbfa46b4498e1732410743e46d6eb0cb
8 years ago
Andrea Frittoli (andreaf) 871f5daa04 Use relative path in data_files in setup.cfg
The absolute path causes failures when installing in a virtual
environment, not having root privileges.

This change solves the installation in a virtual environment,
but it does not fix yet the tempest init workflow when installing
into a virtual environment.

Partially-fixes bug #1480389

Change-Id: Iddefb8db77ffe35a84fb35b6f9e7e823de4ab45f
8 years ago
Matthew Treinish f610aca040
Add tempest init command to tempest cli
This commit adds a new 'tempest init' command to the unified cli.
This command tempest init will be the first step of the new workflow
for running tempest. It is used to initilize a local working directory
to run tempest from. The idea being you can setup different local
working dirs for different tempest envs to test against several
different clouds.

The basic idea for this new workflow after installing tempest is
something like:

 1. mkdir tempest-newcloud
 2. cd tempest-newcloud
 3. tempest init
 4. edit local config file
 5. tempest run

The next step after this is to start working on the 'tempest run'
command. (which will include the hooks for the tempest plugin
interface) But, until that is added a test runner can be called
directly inside a local tempest working directory.

Partially Implements bp tempest-cli-improvements

Change-Id: I572267e61ba8ca86f92299e174864a7a89597207
8 years ago
Matthew Treinish d450a19b3a Enable tempest install to setup a config dir
This commit adds the necessary bits to tempest's setup.cfg file to
instruct pbr on how to make sure that the sample config files and
a /etc/tempest dir get setup when installing tempest. In the new
cli workflow it is expected that by default /etc/tempest will exist
as a source for local working dir configs.

Partially implements: bp tempest-cli-improvements

Change-Id: I2bf93f6789601caaebdfb85ac2b9f34b82f87980
8 years ago
David Paterson 4af1d02746
Add cliff based common cli entrypoint
This commit adds the basic framework to tempest for a unified modular
CLI endpoint using cliff. As of right now this entry-point doesn't
really do anything but exist. It will be expanded in the future first
all the existing commands will be refactored to be used through this.
Secondly a new interface and workflow for running tempest will be
created on top of this.

Co-Authored-By: Matthew Treinish <>

Change-Id: I1ff8d22c120dbc81d812f1f107db7c2d9b15a505
Implements: blueprint tempest-cli-improvements
8 years ago
Jenkins f728beca05 Merge "Tool for generation user accounts from spec" 8 years ago
sslypushenko 0de7d05963 Tool for generation user accounts from spec
This tool can be used for creating user accounts for
tempest runs in concurrency.

Basic usage:
	--tag foo
        -c tempest.conf
	--os-username <admin-name>
	--os-password <admin-password>
	--os-tenant-name <admin-tenant>
	-r <concurrency count>
	--with-admin (optional)

Change-Id: I3c313787b0f6f69edcc5ad16c6c186226f619692
8 years ago
Matthew Treinish 3460aaaf3e
Add py34 to readme, pkg metadata, and envlist
This commit adds references explaining the current state of python 3.4
and tempest to the readme, pkg metadata, and adds an etry to the tox
envlist for running the unit tests on python 3.4.

Change-Id: I9c1e3cdf53dbae723a9fee67ad96932709d2305f
8 years ago
Matthew Treinish d96a6b04ea
Remove version string from the setup.cfg
PBR handles setting the version based on the git tags, so we don't need
to do this in the setup.cfg anymore.

Change-Id: I3c12f54682b58c9eea5b40fa2c34979775c0b827
8 years ago