Commit Graph

548 Commits (0ea2c01d757375d54aa9a9de34c2c9995bb48d13)

Author SHA1 Message Date
Martin Kopec 598b1ae854 Improve router deletion logging in tempest cleanup
Improve logging in tempest cleanup when a router and
its interfaces are being deleted. The review adds a
try except block for every attempt to remove a port.
By that addition any port error will not be hidden
behind the router error the port is attached to.
It will provide more precise error logging.

Change-Id: I475deec7b29600627f68ff07c5546e55cdab100f
4 years ago
Martin Kopec fd01fe9445 Improve logging in tempest cleanup
Add id or name of the resource which failed to delete
to the logging.

Change-Id: I04b04406e3917a63332199adf5c15b8e4ed6ca5a
4 years ago
Manik Bindlish 21491dff81 Checking config file actually exist or not in tempest run
This PS will check config file passed from command line is actually
existing or not in tempest run command. If file exists, only then it
will go for set config path otherwise will fail with value error.
Also adding and modifying unit test cases for this change.

Partially-Implements: blueprint tempest-cli-unit-test-coverage

Change-Id: I09d756be69cb3a9be8d0638c41d45a089d62b238
Closes-Bug: #1808473
4 years ago
Zuul 67e0778618 Merge "Add autopep8 to tox.ini" 4 years ago
Zuul 31db1f6f39 Merge "Update json module to jsonutils" 4 years ago
afazekas 40fcb9ba28 Add autopep8 to tox.ini
autopep8 is an automated code formatting tool for python,
it does not know everything, also not super fast,
but it is faster than I can manually reformat the code.

tools/ will call the formatting,
the tox will check did you called it.

Also adding a tox -eautopep8 way to use it.

autopep8 formats the lines in one way,
where tempest used a another way it was changed.

Change-Id: I6d51b14a5a5b87761071d0927fca23ba1651aa41
4 years ago
zhulingjie 92b87a5a43 Update json module to jsonutils
oslo project provide jsonutils, and we use it in many place[1],
this PS to update the remained json moudule to oslo jsonutils for

[1]: 9594233ffd/tempest/lib/services/volume/v1/ (L15)

Change-Id: Ib8d9f49a1fdfa0f5e0cfb05a193dffb83e4cfe69
4 years ago
Masayuki Igawa 13c588373c
Set concurrency default value
This commit sets the default value 0 for tempest run command.
Because stestr's run_command() requires an int value for that, but we
passes `None` if users don't specify a concurrency value. We found that
behavior with recent stestr change[0]. But we also have the bug in
tempest side.


Change-Id: I83f76acb6a106f7401904a910a5dced47c2db839
4 years ago
Tony Breeds a7c4c14493 stestr expects --concurrency to be an int
Change-Id: Ia56c182287510ffbaee412b6e2a3b1eb5ad510e0
4 years ago
Martin Kopec e681998023 Add NetworkSubnetPools to tempest cleanup
The review adds support for NetworkSubnetPools service,
so if there are leftover subnet pools, tempest cleanup
is able to detect them and remove eventually.

Change-Id: Ieecde490d5eb20e1a894a7bdf3bcf0e7a54c08e2
Related-Bug: #1812660
4 years ago
Martin Kopec 5a884bfbb0 Fix tempest cleanup
Edit service classes so that they discover initial
state of resources before deleting any.

Unify service names - f.e. if a service returns resources
in a list named server_groups, server_groups should be
the key of initial resources in the saved_state.json.

When is_preserve is True, security groups in
NetworkSecGroupService were filtered by networks present
in tempest.conf, however, these groups are associated
with a project_id, therefore it should be filtered
against projects present in tempest.conf.

Change-Id: I97d0115bbb43a089b33602df7c98e153984ceaf1
Related-Bug: #1812660
4 years ago
Martin Kopec 9e43fd8bbe Remove deprecated services from cleanup
The patch removes deprecated services from
 * NetworkVipService
 * NetworkMemberService
 * NetworkHealthMonitorService
 * NetworkPoolService
 * FloatingIpService
 * SecurityGroupService
The above services are not used by the cleanup tool, they
call clients which were already removed from Tempest or
the clients are marked as deprecated ones.

Change-Id: I55ddbce64404c67688600dc6b1231d0bd8ff7006
Related-Bug: #1812660
4 years ago
Martin Kopec 6caf3fabbb Fix tempest cleanup for python3
When using python3 the file which is read and
loaded by jsonutils needs to be opened in binary
mode, otherwise, it's causing problems.
It is a known bug [1] in jsonutils from
oslo.serialization module.
The patch also adds a unit test which tests
the loading of a json file.


Closes-Bug: #1806089
Change-Id: I3db71c6662d9137a4b75381de7d6dbddd3de4e09
4 years ago
Zuul 38e397f64a Merge "Fix tempest cleanup AttributeError" 4 years ago
Zuul 139b453d00 Merge "Fixing ignored value of --config parameter in tempest account-generator" 4 years ago
Martin Kopec b37903cddd Fix tempest cleanup AttributeError
It seems cleanup_service still uses deprecated
tenant_id variable, the patch fixes that.

Closes-Bug: #1812587
Change-Id: I294a2c04e70b980e0f42a29efa7f7448c31d1dc4
4 years ago
Martin Kopec 07a572ca17 Fix verify-config KeyError
When a user runs verify-config while custom, third
party plugins are available in the system, it fails
with KeyError.
The patch fixes that by informing a user about the
plugins and suggesting their setup.

Related-Bug: #1812385
Change-Id: I8019a82717d9463bdc1a6e5025758e1f917358bb
4 years ago
Manik Bindlish 3e99c04329 Fixing ignored value of --config parameter in tempest account-generator
Config parameter of tempest account-generator was ignored.
Instead of that default location of config file were used.

If tempest.conf file doesn't exist on default location
then command was failing.
It never used the config file value from command line.
Command was running successfully only when running from tempest
directory or tempest.conf exist in current directory

Test cases from below patches are relevant for this patch:


Partially-Implements: blueprint tempest-cli-unit-test-coverage

Change-Id: Ic8dcaff920e5cdfb9a9457ff3a3137c225622409
Closes-Bug: #1766354
5 years ago
Zuul f9650269a3 Merge "Add missing ws seperator between words" 5 years ago
Manik Bindlish 6c95678a76 Handling invalid path of workspace register and move
This PS will fix the invalid value handling.
Error will be raised if no/blank value will be specified for
workspace register and workspace move for path parameter.

Change-Id: I0d9956cac27fd4dbb527fd865aa152e4724c01f9
Closes-Bug: #1799883
Partially-Implements: blueprint tempest-cli-unit-test-coverage
5 years ago
zhufl de67637682 Add missing ws seperator between words
This is to add missing ws seperator between words, usually
in log messages.

Change-Id: I228573a755717bb9725267fad61706ce2ef3da87
5 years ago
Zuul d6ffc93c2b Merge "Handling invalid name of workspace register and rename." 5 years ago
Manik Bindlish 864f37eb66 Handling invalid name of workspace register and rename.
This PS will fix the invalid value handling.
Error will be raised if no/blank value will be specified for
workspace register name and workspace rename.

Change-Id: Id8a3f496a8902bd7f0d66254ee8fa148675974e9
Closes-Bug: #1791007
5 years ago
Manik Bindlish a794e85304 Removing redundant spec value in
There is redundant value used for spec in
There is no use of duplicate value as it makes the
code inconsistent.

This patch set will correct the code and also the test
cases which are wrongly designed as per redundant code.

Change-Id: I4bd31f7bb88943b0981c7081f82d80aa28dcdfa4
5 years ago
Manik Bindlish d8e6135c3a Removing redundant "if" from
There are 2 same if conditions: "if" used together
It creates unnecessary redundancy.
This patch set will correct the code and remove one of them.

Change-Id: I1a452b11255346f997832bf18c2f6afc635d5706
5 years ago
Zuul ecddd38ecb Merge "Set max_microversion 2.35 for nova image tests" 5 years ago
Vishakha Agarwal 823450339e Removing unnecessary else
The patch addresses the removal of 'else'
written for if which is irrevalent.

Change-Id: I538ae1380681ec1b0feb0505142549a2ba4eb458
5 years ago
Martin Kopec d7e05dd767 Prevent verify-config from IndexError
verify-config raises IndexError when tempest.conf
contains an empty api_extensions list.
The patch fixes that and adds a few unit tests to

Change-Id: Ia9abea77e3baeb9418cb1869cec2a159cdc9fd48
Related-Bug: #1786915
5 years ago
Ghanshyam ff2c986da2 Remove deprecated option volume_feature_enabled.api_v1
volume_feature_enabled.api_v1 config option to enable the
Cinder v1 API was deprecated in Juno

Cinder v1 API tests were also got removed from Tempest in Juno

This commit remove the deprecated api_v1 config option and also
remove the cinder service clients alias from

Note: cinder v1 service clients alias are being used by
many plugins (depends-on patches). These are being used mainly in
scenario manger copy of plugins.

Next we will deprecated the cinder v1 service clients present in

Only networking-fortinet left to merge the changes which seems inactive
now a days-

As reminder sent in ML, we are good to merge this patch

Change-Id: Ief961758ee413d69d89d45a1b36f1d126c6101cf
5 years ago
ghanshyam 6c682ffc5d Make volume client alias with _latest suffix
This is next step patch from
which makes all the volume service clients alias with _latest suffix.

As all the volume clients point to v3 service client only, having
alias with _v2_ or _v3_ name are confusing.

This commit provide alias with _latest suffix and use them in
tempest. As old alias are being used in tempest plugins also,
we keep the old alias also and slowly will move each plugins to
start using new alias.

Change-Id: Ib89ab72ab4ada3e8442901f104b0b96072187ef9
5 years ago
zhufl 66275c2496 Set max_microversion 2.35 for nova image tests
1. The compute images API is deprecated from the Microversion 2.36,
       so we should set max_microversion = '2.35' for compute image tests.
    2. Compute image_create is not deprecated.
    3. In testcases other than those in compute/images, we can simply
       change to use glance images_client, which will also work before
       microversion 2.35.


Implements blueprint: clear-deprecated-api

Change-Id: I7a40d26449a361f0e3ec922ac41a7c522b572803
5 years ago
Stephen Finucane 7f4a6211bf Mark raw strings as such
This is raising warnings now and will be an error in future versions of
Python 3.

Change-Id: I8827cb36ef2ef85e8f245a2a181fb23ee75bfd16
5 years ago
baiwenteng 781fe07452 Fix typos
replace seperate with separate

Change-Id: I22b0e9d889a507107e853777ec6cfa56fb93d7aa
5 years ago
wangqi 814a87ca6f Modify the empty list ensure method
This commit just simplifies the condition.
The original code is not wrong but this
patch makes it simpler than the original.
It would be better for the maintainance code.

Change-Id: I00e5b75290f10e86c8e45b5d224ef66d7938bb69
5 years ago
Federico Ressi 2d6bcaa7e5 Fix code to pass pep8 Tox environment.
Command tox -e pep8 is failing making CI to fail: this fixes it.

Change-Id: I5b1acfc4ba2e0eb16f50bfd1cdaa8edc361923d1
5 years ago
Doug Schveninger 8b3dc8619b Add verbose option for stdout in subunit_describe_call
The PS will allow request and response header and
body data to be printed to stdout when the -v or
--verbose switch is given.

Change-Id: Ibc3ac0ee717e6f71e74fd5970877a7eda89be3a7
Closes-bug: #1749238
5 years ago
Masayuki Igawa ff07eace4c
Add unit test for tempest run --config-file
This commit adds a unit test for `tempest run --config-file`. That
if block was not tested in the gate. And an issue occurs like
this[0]. And this commit fixes a couple of issue in through
adding this test.


Change-Id: I235b822d802cf27d2dc43b35f85802d3a214b1b5
5 years ago
ghanshyam 2998117ba6 Fixing the condition to run tests in tempest run
During stestr migration, it  seems like we stopped running the
tests in tempest run cli if config_file or workspace arg are passed.

This commit fixing this behavior.

Change-Id: Idf77ff9dd9c0ba0a134dfe61b1ab79d069d3ff0c
5 years ago
Matthew Treinish 138692085d Add a black_regex flag to tempest run
Now that we're using stestr internally the selection regex is more
sophisticated, and lets you exclude on match from a cli regex too. To
leverage this functionality this commit adds a new cli argument to
use it.

Change-Id: I440ab90f576e1ac333c7c40175ad6117d1299c04
5 years ago
Matthew Treinish f9902eca55
Fix the -l/--list-tests argument for tempest run
The tempest run to stestr migration accidently removed the list command.
This commit adds the functionality back and adds a unit test to ensure
this doesn't happen in the future.

Change-Id: Ibc74887dec98f5d9adce56c235d28b19d0a94a63
5 years ago
Matthew Treinish 3e97aae829
Cleanup selection list args in tempest run
The selection list arguments had a mutually exclusive flag set for no
reason. There previously was a limitation in how the flags were used
previously which made those flags mutually exclusive, however that
hasn't been the case for some time. At the same time the docs on
selection needed to be updated, because they showed incorrect internal
details (even prior to the stestr migration) for the blacklist file.
Instead of trying to detail the entire explanation, it now just links
to the stestr docs.

Change-Id: I90dfc15765057f876ebaf66abf5af24de0f675b9
5 years ago
Zuul beb7716bb4 Merge "Remove leftover call to _create_stestrepository" 5 years ago
Adam Harwell e377cab5ed Remove leftover call to _create_stestrepository
This method doesn't exist and causes runs to fail.

Change-Id: I639477e3702029e86509bdf5ae2826e2069a4288
5 years ago
Zuul 24a753dcfb Merge "Switch Tempest CLI commands from testrepository to stestr" 5 years ago
Chandan Kumar 8a4396e3d3 Switch Tempest CLI commands from testrepository to stestr
This commit switches the Tempest CLI commands to internally use stestr
instead of testrepository. At this point in time the testrepository
project is effectively unmaintained and stestr was a fork started to
have an actively maintained test runner. It also focuses on being a
dedicated python test runner, instead of an abstract test runner
interface for any tests that emit subunit.

Besides the bug fixes and other improvements included with stestr, this
switch provides a number of advantages for tempest. Primarily stestr has
a real python API for invoking the test runner directly from python. This
means we can simplify the wrapper code to simply call a function instead
of building out a set of CLI arguments and passing that to the CLI

Co-Authored-By: Matthew Treinish <>

Depends-On: Ic1fa3a98b6bcd151c489b078028687892655a19b
Depends-On: I3855aad5ce129ec8ccb87c05f7aa709b74070efe
Change-Id: I6f5fa7796c576b71c4a0dde66896974a8039a848
5 years ago
ghanshyam 6283daa5ad Remove deprecated 'orchestration' config group options
'orchestration' config group options were deprecated in

And heat tests also been removed from tempest
- Ifc2cac575919da4e361eaa3234d7e0f3e2f98d21

Now we can remove these deprecated config options.

Cleaning up heat related things in account_generator,
cleanup_service etc.

Change-Id: I1c02fd88c3d93b511ff4a6b54b668ebd306481a9
5 years ago
Guoqiang Ding e798bccc3c Fix the mistake of code indent
Change-Id: Ib13e25a25d594935db515e2dce9989bbba024e01
6 years ago
Masayuki Igawa bbbaad68c3
Fix docs markup consistency
This commit fixes docs consistency about markups, mainly. Originally,
some of command options were written with strong emphasis. However,
double dashes are converted to a single dash with it. And this commit
also fixes some inconsistencies and weird indentations.

Change-Id: Iff1e8e320dcb1fa69ca0fce139c58727fca7b729
6 years ago
Zuul 0f7071b948 Merge "Edit documentation for account-generator" 6 years ago
Martin Kopec 14e0be1a74 Edit documentation for account-generator
Add locations which are searched for tempest.conf by default.
--concurrency argument is optional as it contains a default value.

Change-Id: Ie5ca2beb1997e872bfb6370c93869e2ff143f6b3
Related-Bug: #1624066
6 years ago