the purpose of this change is to enable creating advance network
scenarios like checking connectivity from VM with port type "direct" to
VM with port type "normal".
with this change scenario planer can overwrite tempest.conf vnic_type
and port_profile paramters in scenario (ex. connectivty between VM
"direct" and VM "normal").
Change-Id: I5b9a4bd5350809d6b295dfede0c5059d01455404
autopep8 is an automated code formatting tool for python,
it does not know everything, also not super fast,
but it is faster than I can manually reformat the code.
tools/format.sh will call the formatting,
the tox will check did you called it.
Also adding a tox -eautopep8 way to use it.
autopep8 formats the lines in one way,
where tempest used a another way it was changed.
Change-Id: I6d51b14a5a5b87761071d0927fca23ba1651aa41
Below nodesets defined in Tempest are removed now:
- openstack-bionic-node This is unused after I5a24c697103090f6712d6e9168abe6c86f7c380c
- openstack-opensuse150-node is duplicate of devstack-single-node-opensuse-150
Depends-On: https://review.openstack.org/#/c/641746/
Change-Id: I39c03bfa52191a0b1acf6a73fa31b3c071ee60c1
This commit makes tempest-tox-plugin-sanity-check job
to run on Bionic. This job only run on master Tempest gate
so no need to worry about its xenial version.
Change-Id: I6d8c63a62416b35e9d203cbeabc9868aa6a95f44
oslo project provide jsonutils, and we use it in many place[1],
this PS to update the remained json moudule to oslo jsonutils for
consistency.
[1]: 9594233ffd/tempest/lib/services/volume/v1/quotas_client.py (L15)
Change-Id: Ib8d9f49a1fdfa0f5e0cfb05a193dffb83e4cfe69
This commit sets the default value 0 for tempest run command.
Because stestr's run_command() requires an int value for that, but we
passes `None` if users don't specify a concurrency value. We found that
behavior with recent stestr change[0]. But we also have the bug in
tempest side.
[0] https://github.com/mtreinish/stestr/issues/231
Change-Id: I83f76acb6a106f7401904a910a5dced47c2db839
No need for dynamic credentials to test showing the default domain.
By setting this value to False, consumers with an immutable user source
can execute this test.
Depends-On: I83a9b8af775580d36a1141be55e9c1cc283a75b6
Partial-Bug: #1714277
Change-Id: Ib85691ae3f7b5a4d4a9da620b6ec46c44380ef03
tempest-slow has been added for stable branches also
- I5cb829beebcc59de682cba927f5b01793ec70d00
let's make this job definition to allow to run on stable branches also.
Change-Id: Ie5492603a12ca4ae56baa744bf650ae4d8036256
tempest-multinode-full should be available to run on
all stable branch not just master. All the devstack
changes are backported already.
tempest-multinode-full will use Bionic node from
stein onwards and xenial for all other stable branches.
Change-Id: I94f66028df58a69aa7c26e8a03428bf48596d25c
Some systems require strong passwords, so we'd better
use data_utils.rand_password() to create password when
creating user, to avoid password strength validation error.
Change-Id: I503ba6e068b6f7c8487b7077637ee21a9c104595
* To use a fresh venv, use `--clear` option when creating a virtualenv
* Removes the project as the script fails to uninstall the dependencies.
* Added missing tempest plugins
* Log the stdout of each plugin in a file
* Adds some plugins to BLACKLIST to avoid errors
Change-Id: I5068d96105a22cbfd33f5900f4bdc21b47a65c06
The review adds support for NetworkSubnetPools service,
so if there are leftover subnet pools, tempest cleanup
is able to detect them and remove eventually.
Change-Id: Ieecde490d5eb20e1a894a7bdf3bcf0e7a54c08e2
Related-Bug: #1812660
In the previous patches, tempest cleanup got improved
and new methods were implemented. This review adds
more unit tests to exercise those changes and to
improve tempest cleanup test coverage.
Change-Id: Ibf30162e49a8cf87accdbe7f0a6cc38941873d5e
Related-Bug: #1812660
Edit service classes so that they discover initial
state of resources before deleting any.
Unify service names - f.e. if a service returns resources
in a list named server_groups, server_groups should be
the key of initial resources in the saved_state.json.
When is_preserve is True, security groups in
NetworkSecGroupService were filtered by networks present
in tempest.conf, however, these groups are associated
with a project_id, therefore it should be filtered
against projects present in tempest.conf.
Change-Id: I97d0115bbb43a089b33602df7c98e153984ceaf1
Related-Bug: #1812660
The primary goal is to be able to run all Tempest tests
and verify OpenStack when profiling is enabled. Also this patch
allows to:
* manually verify that certain services are properly instrumented
and produce trace events when a scenario is executed;
* write automatic tests for trace coverage;
* profile certain tests from performance perspective.
A new parameter is introduced into tempest.conf:
* profiler.key - the key used to enable OSProfiler (should
match the one configured in OpenStack services)
To test the patch on DevStack:
1. Enable osprofiler with Redis collector in local.conf:
enable_plugin osprofiler https://git.openstack.org/openstack/osprofiler master
OSPROFILER_COLLECTOR=redis
2. Run all Tempest tests or select some, e.g.:
tempest run --regex tempest.api.network.test_networks.NetworksTest.test_list_networks*
Change-Id: I64f30c36adbf7fb26609142f22d3e305ac9e82b5
I don't see any limitations by using pre-provisioned
credentials for these tests:
* test_group_create_update_get
* test_group_users_add_list_delete
* test_list_user_groups
* test_list_groups
Change-Id: Id22911035ce880ab1faa9b37b238b9372ae98087
Based on average test times from the last 300 runs
of the tempest-full job [1] this change marks the
test_attach_detach_volume test as slow but only if
SSH validation is enabled, which it is in tempest-full
but might not be in other jobs that run it, for example
in refstack which uses this test for interoperability
certification.
[1] http://paste.openstack.org/show/745114/
Change-Id: I47710321d8dfb0345c793e66bd09a5dd1933557b
Related-Bug: #1783405
There are cases where we want to conditionally apply an
attribute to a test function, for example, if SSH validation
is enabled then a test may run much slower than if it is not.
This adds a 'condition' kwarg to the attr() decorator which
behaves similarly to the 'condition' kwarg on the skip_because()
decorator.
Change-Id: I83233854a217b6961e7614d7d9df1b4fc8d5a640
Based on average test run times from the last 300 runs of
the tempest-full job [1] the test_server_connectivity_reboot
test is third slowest at about 200 seconds. This should not
be a surprise given all of the other tests in the same class
are already marked slow.
This change marks that test slow like the others which means
it will be run in the tempest-slow job.
[1] http://paste.openstack.org/show/745114/
Change-Id: Ie8ea1555cc4512bf29dff1e7df592dedfab28c61
Related-Bug: #1783405
Based on the average test times taken from the last 300 runs
of the tempest-full job [1] the test_volume_boot_pattern
scenario test is the slowest test that is not marked slow at
208 seconds.
This change marks that test as slow which means it will still
be run in the tempest-slow job.
[1] paste.openstack.org/show/745114/
Change-Id: Ibc5e5ecaa997744aecea8aff07e7a7e06178f5f3
Related-Bug: #1783405
The patch removes deprecated services from
cleanup_service.py:
* NetworkVipService
* NetworkMemberService
* NetworkHealthMonitorService
* NetworkPoolService
* FloatingIpService
* SecurityGroupService
The above services are not used by the cleanup tool, they
call clients which were already removed from Tempest or
the clients are marked as deprecated ones.
Change-Id: I55ddbce64404c67688600dc6b1231d0bd8ff7006
Related-Bug: #1812660
