In the previous patches, tempest cleanup got improved
and new methods were implemented. This review adds
more unit tests to exercise those changes and to
improve tempest cleanup test coverage.
Change-Id: Ibf30162e49a8cf87accdbe7f0a6cc38941873d5e
Related-Bug: #1812660
Edit service classes so that they discover initial
state of resources before deleting any.
Unify service names - f.e. if a service returns resources
in a list named server_groups, server_groups should be
the key of initial resources in the saved_state.json.
When is_preserve is True, security groups in
NetworkSecGroupService were filtered by networks present
in tempest.conf, however, these groups are associated
with a project_id, therefore it should be filtered
against projects present in tempest.conf.
Change-Id: I97d0115bbb43a089b33602df7c98e153984ceaf1
Related-Bug: #1812660
I don't see any limitations by using pre-provisioned
credentials for these tests:
* test_group_create_update_get
* test_group_users_add_list_delete
* test_list_user_groups
* test_list_groups
Change-Id: Id22911035ce880ab1faa9b37b238b9372ae98087
Based on average test times from the last 300 runs
of the tempest-full job [1] this change marks the
test_attach_detach_volume test as slow but only if
SSH validation is enabled, which it is in tempest-full
but might not be in other jobs that run it, for example
in refstack which uses this test for interoperability
certification.
[1] http://paste.openstack.org/show/745114/
Change-Id: I47710321d8dfb0345c793e66bd09a5dd1933557b
Related-Bug: #1783405
There are cases where we want to conditionally apply an
attribute to a test function, for example, if SSH validation
is enabled then a test may run much slower than if it is not.
This adds a 'condition' kwarg to the attr() decorator which
behaves similarly to the 'condition' kwarg on the skip_because()
decorator.
Change-Id: I83233854a217b6961e7614d7d9df1b4fc8d5a640
Based on average test run times from the last 300 runs of
the tempest-full job [1] the test_server_connectivity_reboot
test is third slowest at about 200 seconds. This should not
be a surprise given all of the other tests in the same class
are already marked slow.
This change marks that test slow like the others which means
it will be run in the tempest-slow job.
[1] http://paste.openstack.org/show/745114/
Change-Id: Ie8ea1555cc4512bf29dff1e7df592dedfab28c61
Related-Bug: #1783405
Based on the average test times taken from the last 300 runs
of the tempest-full job [1] the test_volume_boot_pattern
scenario test is the slowest test that is not marked slow at
208 seconds.
This change marks that test as slow which means it will still
be run in the tempest-slow job.
[1] paste.openstack.org/show/745114/
Change-Id: Ibc5e5ecaa997744aecea8aff07e7a7e06178f5f3
Related-Bug: #1783405
The patch removes deprecated services from
cleanup_service.py:
* NetworkVipService
* NetworkMemberService
* NetworkHealthMonitorService
* NetworkPoolService
* FloatingIpService
* SecurityGroupService
The above services are not used by the cleanup tool, they
call clients which were already removed from Tempest or
the clients are marked as deprecated ones.
Change-Id: I55ddbce64404c67688600dc6b1231d0bd8ff7006
Related-Bug: #1812660
oauthlib uses the request URI as input for generating an oauth
signature and matching it against the incoming oauth_signature request
attribute. tempest also uses the URI to generate this signature, which
means the URIs must match exactly.
If the keystone catalog contains an endpoint with a trailing slash, such
as https://keystone.example.org/v3/, then the URI that tempest uses for
the signature generation will be
https://keystone.example.org/v3//OS-OAUTH1/request_token. The incoming
request URI that oauthlib sees will not have the duplicate slash and so
the resulting signature will differ, and the auth request will be
rejected due to mismatching HMAC-SHA1 signatures.
This patch corrects the issue for deployments that use a trailing slash
in their keystone catalogs (which is valid) by normalizing the path
before generating the signature.
Change-Id: Ie827b1af7b23c6d6eaf2c2894cc2629d0e252108
I don't see any limitations by using pre-provisioned
credentials for these tests:
* test_project_create_with_domain
* test_project_create_with_parent
* test_create_is_domain_project
* test_project_get_equals_list
* test_project_create_with_description
* test_associate_user_to_project
* test_project_create_enabled
* test_project_create_not_enabled
* test_project_update_name
* test_project_update_desc
* test_project_update_enable
By setting force_tenant_isolation=False these tests now be
can executed with backends that don't allow user creation
(immutable user source) like LDAP.
Change-Id: Ifc9f23d150992cdcf415293439c2f51f3d75aeb3
If the keystone user source is immutable, such as an LDAP
active directory implementation, tempest tests that try
to create or delete a user will fail. Instead of failing,
we would like them to skip. This change uses a testtools
decorator to avoid unnecessary modifications and allow those tests
to skip.
Partial-Bug: #1777047
Change-Id: I8a0d5935b695a1649b204bd3c3351f34a80e9059
I don't see any limitations by using pre-provisioned
credentials for this test:
* test_list_regions_filter_by_parent_region_id
* test_create_region_with_specific_id
* test_create_update_get_delete_region
* test_list_regions
By setting force_tenant_isolation=False this test can now
be executed with backends that don't allow user creation
(immutable user source) like LDAP.
Change-Id: Ifad293dddb7c7fe76005761b43e1252e40ce7bdc
When using python3 the file which is read and
loaded by jsonutils needs to be opened in binary
mode, otherwise, it's causing problems.
It is a known bug [1] in jsonutils from
oslo.serialization module.
The patch also adds a unit test which tests
the loading of a json file.
[1] https://bugs.launchpad.net/oslo.serialization/+bug/1515231
Closes-Bug: #1806089
Change-Id: I3db71c6662d9137a4b75381de7d6dbddd3de4e09
The volume multiattach tests originally required special
devstack configuration in Queens for the Ubuntu Cloud Archive,
but that is no longer necessary. This change enables the volume
multiattach tests in the tempest-full(-py3) and tempest-slow jobs
so we can drop the nova-multiattach job, which is mostly redundant
coverage of the tempest.api.compute.* tests, and reduce the total
number of jobs we run against nova/cinder/tempest changes.
Due to intermittent bug 1807723 when running the
test_volume_swap_with_multiattach test with two compute services,
that test is conditionally skipped if there is more than one compute.
This is probably no great loss in test coverage for now given
swapping multiattach volumes is likely rarely used (see bug 1775418).
Related ML thread:
http://lists.openstack.org/pipermail/openstack-dev/2018-October/135299.html
Depends-On: https://review.openstack.org/634977
Change-Id: I522a15ba3dbfee5d8ef417e43288a12319abf6ff
'interop-tempest-consistency' job
has been added to check the test renaming
compatibility with interop usage.
This job was added as n-v to check the stability
and it is running fine since it was added so
making this as voting job in check pipeline.
Change-Id: I088f64d87b04b5db2319ccd3b3d02821a06f35e9
