14176 Commits

Author SHA1 Message Date
Lee Yarwood
9ba15f64bc compute: Move device tagging tests to use wait_until=SSHABLE
Waiting until the test instance is SSH-able before continuing
the test will help us avoid failures to detach a volume from
server, more info in the related bug.

Related-Bug: #1960346

Change-Id: Id5496572ce6cef06e71bae1a1040b274bacf4f79
2022-03-02 09:52:48 +00:00
Lee Yarwood
d548e7a8fb compute: Move volume attach tests to use wait_until=SSHABLE
Waiting until the test instance is SSH-able before continuing
the test will help us avoid failures to detach a volume from
server, more info in the related bug.

Related-Bug: #1960346
Change-Id: I5ad4aa04f02001efa06616bf77e03e018206d1fe
2022-03-02 09:46:42 +00:00
Lee Yarwood
0b4bc3dcc6 Introduce PINGABLE and SSHABLE waiters and wait_until state support
This change introduces two new instance state waiters and uses them to
extend the existing tempest.common.compute.create_test_server wait_until
state support.

They are being introduced in an effort to allow the guest OS time to
start *before* we start attempting to interact with it either directly
by connecting to the instance or indirectly by hot-plugging or
hot-unplugging devices. The latter on some virt backends being an issue
if the guest OS is unable to respond to the underlying ACPI requests
sent to it.

It should be noted that these new states rely on the instance already
being ACTIVE before we begin to wait for the instance to either become
pingable or accessible over SSH. This is taken into account and will
always happen for these states even if validation isn't enabled in the
environment and thus it isn't possible to wait until the instance is
pingable or accessible over ssh.

Change-Id: Ib14fa7dc5c8093eed498049cd0a56c8ac6853660
2022-03-02 09:21:16 +00:00
Zuul
ca03d2b504 Merge "Move get_server_ip into tempest.common.compute" 2022-02-17 11:19:22 +00:00
Zuul
42f6cb522e Merge "Add logging to metadata parsing" 2022-02-14 04:28:29 +00:00
Zuul
165397af71 Merge "Fix floating ip timeout issues with uwsgi enabled" 2022-02-10 10:37:46 +00:00
Zuul
140d693f85 Merge "Remove usage of unittest2" 2022-02-09 20:42:29 +00:00
Martin Kopec
78b960f91f Fix floating ip timeout issues with uwsgi enabled
This reverts commit d4c9cc89f199899b20cfaea975ecd15ee66d3523 which
introduced a waiter to fix a bug in Tempest. Unfortunately that broke
neutron-tempest-with-uwsgi job where some of the tests started to fail
due to timeouts - waiting for the floating ip to get attached to a server
prolonged the tests which caused the timeout (it seems the ip attachment
process takes longer on an environment with uwsgi).
The failing tests also didn't require an attached floating ip or they
did however long enough after the server creation thus it is redundant
to wait for the ip attachment at the beginning (when the resource is
created) when it can happen "in parallel".

This patch moves the waiter from the common file to the test class
where the waiter is needed to fix a bug [1].

[1] https://bugs.launchpad.net/tempest/+bug/1945495

Closes-Bug: #1960022
Change-Id: Iddfa4ef874328fdee5418de5a8b1112007541883
2022-02-09 16:23:06 +00:00
Ghanshyam Mann
eb1afd539a Make stable/ussuri job voting
stable/ussuri job was made non voting due to
bug#1959467

- https://review.opendev.org/c/openstack/tempest/+/827258

As bug is fixed now, let's revert it to voting again.

Change-Id: I834300ee8d9ee8952284b55f53d97456a1ca6032
2022-02-05 19:50:33 -06:00
Balazs Gibizer
8f68f69ccf Add logging to metadata parsing
The TaggedAttachmentsTest class queries server metadata to see if the
tagging is visible from the guest. However when the check fails it does
not log what was the faulty metadata. To make troubleshooting easier
this patch adds logging for the error cases.

Change-Id: I58f6fefef0f0a9db2f7fdf2a1a0125f616319949
2022-02-04 15:25:11 +01:00
115065a239 Fix TypeError in QoSBandwidthAndPacketRateTests
[1] Added random int to flavor name but that resulted into
TypeError: can only concatenate str (not "int") to str.

Concatenate using % operator to fix it.

Also make tempest-full-ussuri-py3 job non-voting temporary
to unblock gate, can be reverted once [2] is fixed.

[1] https://review.opendev.org/c/openstack/tempest/+/826946
[2] https://launchpad.net/bugs/1959600

Related-Bug: #1959467
Change-Id: I76e15b6b52bb44fec550ead510186c644c4a1c7f
2022-02-02 10:36:55 +05:30
melanie witt
cae966812a Append random int to new flavor name in QoSBandwidthAndPacketRateTests
In a couple of the tests a new flavor is created by appending the
string 'extra' to the existing flavor name. The existing flavor name is
however CONF.compute.flavor_ref, which isn't unique. If two tests that
create the flavor run at the same time, there will be a 409 collision
and one test will fail.

This appends a random int to the new flavor name to prevent conflicts.

Closes-Bug: #1959467

Change-Id: I9928e2be9c749f69eea9a9d4f9e8fe03ef4ef612
2022-01-29 01:14:06 +00:00
Zuul
46fc2e2e41 Merge "Fix test_rebuild_server test by waiting for floating ip" 2022-01-28 20:23:38 +00:00
Zuul
34c97a1b25 Merge "Revert "Check VM's console log before trying to SSH to it."" 2022-01-28 13:20:07 +00:00
Martin Kopec
d4c9cc89f1 Fix test_rebuild_server test by waiting for floating ip
The test could fail on an adresses mismatch when the test server
didn't have floating ip assigned yet at the beginning of the test.

Closes-Bug: #1945495
Change-Id: Ic99ba61d87dfc24a684263d41e0caa4f54d1d263
2022-01-28 08:00:50 +00:00
Zuul
8c4b99e26b Merge "Fix stable/train testing with tempest 26.1.0" 2022-01-27 00:11:02 +00:00
Ghanshyam Mann
5ff5dabd6e Fix stable/train testing with tempest 26.1.0
Tis issue came up before also during we cap the Tempest
version for stable branch tests(the one moved to EM state).

Tempest use the tox constraints file while creating the tempest
virtual env in devstack/lib/tempest, installing the tempest
plugins and running the tests like in run-tempest ansible roles.
If anywhere tox constraints file is used differently then venv
will be recreated and installed tempest plugins will be gone from
that recreated venv so tempest will not be able to run the plugins
tests and fail job.

Current failure: https://aca43ed1a01d95dea0ee-dcdb6cbb330bdac08ffee1284f86c919.ssl.cf2.rackcdn.com/822225/3/check/octavia-v2-dsvm-scenario/f60e4d3/job-output.txt

Earlier I tried to fix it in a generic way by fetching the devstack
localrc - https://review.opendev.org/c/openstack/tempest/+/774913
but this solution does not seems working as we can see it is skipped
in current stable/train testing with tempest 26.1.0

Adding stable/train explicitly in list for using the stable constraint
in run-tempest role also which will be in sync with what devstack/lib/tempest
use. And later I will debug why generic way is not working.

Because of Tempest 26.1.0 and with stestr 2.5.1 (beacause of upper
constraints of stestr 2.5.1 in stable/train) which does not have
new args exclude-regex so let's fallback to old arg if new arg is passed.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/822380
Depends-On: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/825708

Closes-Bug: #1955418
Change-Id: I105c839c38adfba1f9e22123ec3a99ffc0b5b72e
2022-01-25 17:03:31 +00:00
Jon Schlueter
1ff7748623 Remove usage of unittest2
from comments when it was last touched it looks like workarounds for
unittest2 might be able to be dropped.

related: https://github.com/mtreinish/stestr/pull/265

simplify the workaround logic around unittest2 TestCase logic

Change-Id: Ibac9d0c2fa2f30605dd44ee58b84946464ea6449
2022-01-24 17:49:25 -05:00
Ade Lee
86db21ea6a Patch paramiko
Its not clear how long it will take for paramiko to merge
https://github.com/paramiko/paramiko/pull/1928 to allow paramiko to
work in a FIPS enabled environment.

In the meantime, lets patch paramiko so we can merge our FIPS enabled
gates.  When the patch or its equivalent is merged, we can remove this
code.

Change-Id: Ib59647e6aa3dc313ee603ee9d322416179ef615c
2022-01-24 18:31:34 +00:00
Zuul
eb860bca57 Merge "Make broken tempest-full-py3-opensuse15 job only until stable/xena" 2022-01-24 11:46:42 +00:00
Zuul
e7ab86a797 Merge "Update aggregate tests for bug#1907775" 2022-01-24 01:12:01 +00:00
Zuul
2caabcd6a5 Merge "edit check result and make sure the query result correct" 2022-01-22 03:35:21 +00:00
Zuul
2ea105d084 Merge "add schemas members for image" 2022-01-22 03:05:57 +00:00
Zuul
e45e77769e Merge "delete all items of the group type spec" 2022-01-22 03:05:54 +00:00
Ghanshyam Mann
e6b7056d84 Make broken tempest-full-py3-opensuse15 job only until stable/xena
tempest-full-py3-opensuse15 is failing all the time and
opensuse is not tested/supported distro in testing runtime
or devstack anymore. These job is removed from master gate
but it is in use until stable/xena.

- https://review.opendev.org/q/topic:%22remove-tempest-full-py3-opensuse15%22+(status:open%20OR%20status:merged)

Modifying it for until stable/xena only.

Change-Id: Ibe9166091011c08644cba783b7db77124cb17417
2022-01-21 19:42:24 -06:00
Zuul
42bf51c989 Merge "Revert "Revert "Make centos8 stream job non voting for bug#1957941""" 2022-01-21 10:39:13 +00:00
Ghanshyam
5eb51dc061 Revert "Revert "Make centos8 stream job non voting for bug#1957941""
This reverts commit 1e2fe4fb33cd1ef8f0067a3aa2ef8c10c0ba0d2a.

Reason for revert: they started failing again with RETRY_LIMIT  

https://zuul.openstack.org/builds?job_name=tempest-integrated-compute-centos-8-stream&job_name=tempest+tempest-full-py3-centos-8-stream&skip=0

Change-Id: I42f4991d786e74512a666cc12b7901f5effcf10b
2022-01-21 05:56:22 +00:00
Zuul
c676a1a993 Merge "Add test coverage for bug 1943724" 2022-01-20 22:11:17 +00:00
Zuul
c545cb1c7c Merge "Add support for ecdsa keys" 2022-01-19 03:30:16 +00:00
Zuul
d373b9a176 Merge "Revert "Make centos8 stream job non voting for bug#1957941"" 2022-01-18 22:45:54 +00:00
Ade Lee
6ded070b51 Add support for ecdsa keys
In FIPS mode, using RSA keys for ssh is fine as long as SHA-1 is
not used for the signature algorithm.  Unfortunately, the version
of cirros used in OpenStack CI does not have a version of dropbear
that supports SHA-2 signatures.  So, any connections from a FIPS
enabled machine will fail as the cirros instance will only support
ssh-rsa (SHA-1 signatures).

To get around this, we add a new option to specify the key type
(validation.ssh_key_type).  This will allow the addition of other
key types in future if needed.

Tempest now supports 'rsa' and 'ecdsa' key types.

We also add a fips job to the experimental queue to test the usage
of the new key type.

Change-Id: Ib59eb8432fa1a2813b3047955157d1b3d24a55f8
2022-01-18 15:25:38 +00:00
Ghanshyam
1e2fe4fb33 Revert "Make centos8 stream job non voting for bug#1957941"
This reverts commit 0de0ebf0987c2cd92a3a880de822f3616480d2d5.

Reason for revert: Try with workaround merged in https://review.opendev.org/c/openstack/devstack/+/824862

Change-Id: Iafff24bf829da1d399fed4ca16a62f24196cd4af
2022-01-17 22:09:52 +00:00
Balazs Gibizer
4f94ea018d Update aggregate tests for bug#1907775
Co-Authored-By: Ghanshyam <gmann@ghanshyammann.com>

Change-Id: I2d4316a7f3c88cae32b3898744d13de39446a62d
2022-01-17 14:39:54 +01:00
Zuul
268606a720 Merge "Add fields in hypervisor schema for 2.33 and 2.53" 2022-01-17 02:53:57 +00:00
Zuul
207a045619 Merge "Clean up wait_for_resource_deletion message" 2022-01-16 23:31:39 +00:00
Zuul
e414221c32 Merge "wait_for_server_status(): report original request ID if failure" 2022-01-16 21:07:14 +00:00
Zuul
73268fb6b3 Merge "Set default value of --concurrency to 2" 2022-01-16 18:14:16 +00:00
Ghanshyam Mann
0de0ebf098 Make centos8 stream job non voting for bug#1957941
Two test_network_basic_ops tests are failing consistently in
centos-8-stream jobs (nova side tempest-integrated-compute-centos-8-stream
and tempest tempest-full-py3-centos-8-stream jobs).

1. tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_hotplug_nic
2. tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_network_basic_ops

Failure: https://zuul.opendev.org/t/openstack/build/e0db6a9a7ba04e66b0781ba7d259357d/logs

This is blocking nova and tempest gateso making the centos8 stream job
non voting until bug is fixed.

Related-Bug#1957941

Change-Id: Ia8c575b1accf4fa2a25b903bf9bf5a614752e344
2022-01-14 11:08:05 -06:00
Balazs Gibizer
770dc0060f Add test coverage for bug 1943724
Now that bug 1943724 is fixed in neutron we can extend the tempest test
coverage to show that changing the direction of the QoS minimum
guaranteed bandwidth rule on a bound port is supported.

Related-Bug: #1943724

Change-Id: I09e773baedcffc8af5a88e0be1df38d14633ed74
2022-01-14 17:57:40 +01:00
lpiwowar
bd722d366e Set default value of --concurrency to 2
There is a non-negligible number of tests in tempest that need accounts
generated by account-generator with a concurrency of at least 2. Running
these tests with accounts generated with the current default value for
concurrency makes them therefore fail.

This change sets the default value for concurrency to 2 and updates
the documentation for --concurrency parameter.

Closes-Bug: #1948935
Change-Id: Iada980beff9db3c8cd02c84d0d7602264b66ddf8
2022-01-14 09:25:12 +01:00
Artom Lifshitz
da48e4ee3d wait_for_server_status(): report original request ID if failure
If waiting for a server to reach a particular status fails - for
example, a server never reaches ACTIVE after boot - the most useful
piece of information when debugging the failure is the orignal request
ID - in our example, the request ID of the original POST /servers
request.

This patch makes create_server() remember the original server boot
request ID, and wires that value up into wait_for_server_status(), so
that the latter can log it if the wait fails.

Because we have to remain compatible with calls to
wait_for_server_status() that do not pass a request_id, there's some
conditional logic to only log it if it was passed in.

Similar mechanics can, in the future, be added to other areas where
tests can fail, for example for live migration or resizing, or
attaching a port to an server.

Change-Id: I3d5aa41d96f7fa4fc860ed9d1076a247d5fa7bf6
2022-01-13 16:11:05 -05:00
Zuul
fe0ac89a5a Merge "Test Network QoS minimum packet rate scenarios" 2022-01-12 10:38:38 +00:00
Zuul
3d4e888f31 Merge "Add flavor extra spec validation tests" 2022-01-10 12:02:03 +00:00
Eric Harney
941043fcc7 Clean up wait_for_resource_deletion message
Add missing space and punctuation to make this
message look cleaner.

Change-Id: I595291b68d4fef884888668d3e3c2d5d73b0c4d9
2022-01-07 14:22:09 -05:00
Lee Yarwood
20556df7ab Move get_server_ip into tempest.common.compute
This will be required by the new wait_until={PINGABLE|SSHABLE} logic
within the module so move this down from
tempest.api.compute.base.BaseV2ComputeTest.

A version of this is also present in tempest.scenario.manager however
unifying that with this version is left for another change.

Change-Id: Iddfdf48da58320844e265fb1209e25a53d501f93
2022-01-07 09:57:40 +00:00
Lee Yarwood
1a76c2c1c5 Revert "Check VM's console log before trying to SSH to it."
This reverts commit e3405ba808f97eae57f3a60991000afaa34cbe89.

AFAICT wait_for_sshable has never been wired up for any tests either
in-tree or within the wider opendev namespace:

https://codesearch.opendev.org/?q=wait_for_sshable

Remove this before introducing a fresh implementation overloading the
original wait_unil kwarg.

Change-Id: I15b5cdcda387e4868d4431c86ad087f3e3e2c8fe
2022-01-07 09:57:40 +00:00
Zuul
fd00bffb96 Merge "Remove stable/train jobs from master gate" 2022-01-06 03:28:41 +00:00
Rajat Dhasmana
81c91f0ec2 Fix: Schema validation of volume list detail
When doing the attachment of volume with glance host, the
"server_id" field of volume attachment response is null which is
not handled in the schema validation of list volume with detail.
This caused a failure in one of the tempest volume test
"test_unmanage_manage_volume" where we do a list volume with
detail request and one volume (probably from other test running
in parallel) existed with an active attachment to the glance host[2].
We can see that the "server_id" field of the attached volume is null[3].
The "server_id" field is only present when doing the attachment
with nova.
This patch allows the "server_id" to be "string" as well as "null" type
to handle both nova and glance usecase.

[1] https://86aa517b6cb393144dd6-d78b0c94e26e635eac828273888f290f.ssl.cf5.rackcdn.com/822731/1/check/glance-multistore-cinder-import/e52281e/testr_results.html

[2] Body: b'{"volumes": [{"id": "e11beb8b-e170-4c5c-945b-e854c01f85ab",
"status": "in-use",...
, "attachments":
[{"id": "e11beb8b-e170-4c5c-945b-e854c01f85ab",
"attachment_id": "5132d152-54c3-4aa6-90e8-f13d2d1d0028",
"volume_id": "e11beb8b-e170-4c5c-945b-e854c01f85ab",
"server_id": null,
"host_name": "ubuntu-focal-inmotion-iad3-0027876140",
"device": "glance_store",...

[3] "server_id": null

Closes-Bug: #1956340
Change-Id: I92d4d49bf5fe52f3417555260e0771397f18334d
2022-01-04 05:53:03 -05:00
Zuul
c2f5a47cfb Merge "Add --libpath argv option to fix unittest failure" 2021-12-23 15:03:59 +00:00
Zuul
3557e96e71 Merge "Revert "Skip 3.19 test_reset_group_snapshot_status"" 2021-12-21 18:52:55 +00:00