Accidental double restart may cause race condition of service not up in
time when listed by second restart call of reset_node_service.
Resolves: OSPCIX-467
Change-Id: I941673a3cbb6c755254029b8c7b9eb4a456c2d48
This method checks if the Neutron API replies to a generic command
("agent list").
Related-Bug: #OSPRH-2460
Change-Id: I7982bb130e4a8ef7495e6243f656cc4095d35db8
Recently FQDNs were needed for traffic flow tests [1],
but it seems AZ tests are affected as well from this fix.
This change fixes API network agent listing
for 'OVN Controller Gateway agent' to match short hostnames to zones.
Short hostnames used since API may show a mix of short/full names.
[1] 926195: Use long names when getting router gateway | https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/926195
Change-Id: Iafeced62b4ebe50a20f976803fd8a0cdbfeb3330
Use 'unix' rather than 'punix' when running monitoring command
in ovsdbserver pod since 'punix' is a passive, a.k.a. listening
side of a connection.
Fix typo in ovnsb_db.sock path.
Change-Id: Id384e20ff0c786032fff18ca287ecd9bc32db06b
This allows metadata rate limiting tests to run fully,
yet they will be skipped until feature code will be inserted to RHOSO.
The ini filename/location from which configuration pulled and applied
changed in RHOSO, now used instead of 'oc patch' for edpm nodes.
Also service existence check fixed, pointed to strip method instead of
calling it, which mistakenly interpreted as true, so later code included
crc node as node to restart metadata service, results in failure of test.
In metadata rate limiting tests redundant code removed,
and made configuration setting restart service in minimal amounts.
Change-Id: I2b44e2dbf97f87b8fd9b5feee49a634300367218
In order to ensure traffic flow tests behave consistently
on different environments the functions that are retrieving
router gateway chassis should use full names.
Change-Id: Ic8097270271433e68ffee1b8685caf01097d9393
It was found that sometimes an unexpected output returned when
using 'oc rsh' on attempt to get ovsdb command string from db pod.
This was causing an AssertingError and a test failure.
Compared to 'oc rsh' the issue does not happen when using 'oc exec'
command.
Also running ps in the background using setsid helps to avoid
sporadic errors due to [1].
[1] https://github.com/kubevirt/kubevirt/issues/10240
Change-Id: I77530691613ca352ce5e5650b3a2450a7003f459
Method should return boolean according to check,
yet raises error upon expected non-zero exit status.
Change-Id: I328edfabbf0b7ff0df8eadaf3a151aa2ab9fcfe4
Before this patch, the test test_dns_name_after_ovn_controller_restart
used the value from VM's `OS-EXT-SRV-ATTR:host` to search for the
corresponding compute node within the inventory.
Due to recent changes, the compute name has changed in the inventory and
the proper value that needs to be searched corresponds with
`OS-EXT-SRV-ATTR:hypervisor_hostname` instead.
This patch fixes test_dns_name_after_ovn_controller_restart by using the
value from `OS-EXT-SRV-ATTR:hypervisor_hostname`.
Change-Id: I2e928c987c0446b6c8b8a93abcb58122d3173207
Due to recent changes in inventory creation, node names use FQDN and
this may cause issues when deciding whether a node is an OCP node or
not. Example: "compute-0.ocp.openstack.lab" is not an OCP node, but
"ocp" is part of its hostname.
With this patch, the issue with FQDNs is resolved by checking how the
hostname starts.
Change-Id: I75ad4316338effccb67614997909ee7d7ff17c65
With vlan tenant we have mac-mapping configured in ovs db
for compute nodes. These MAC values should be used when
filtering relevant traffic for east_west communication.
Also, introduced separate tunnel_interface config parameter
to define an appropriate value when capturing traffic on
a normal node and on ovs_pod.
Change-Id: Ibef4498604bd9fff7ab537ed276ce59e8a052cd6
In case proxy host on the environment for some reason
does not have routing to the external network
spawn a VM on external network to use it as a proxy
host for sending traffic to the VM under test.
Note: on podified environment we can not ping VM in
DVR test from a local ip since tempest is running on
one of the OCP cluster nodes and FIP-related traffic
will be captured on the gateway node (when only
compute nodes are expected) and thus affecting the
results.
Change-Id: I35def1d0d0e419cbcf552bc6148ed112567aa8f6
A new target_state parameter was introduced into
remote_service_action() function in order to make sure that
a service reached the expected state.
Change-Id: I6fe09ffa86c7cbc703db56a7b72d0cf90b42f9f4
Nodes interfaces discover was introduced in [1] where code
was extended to support tcpdump captures on different type of
nodes (including OCP cluster nodes).
Since then the code was modified to capture traffic on OCP nodes
differently comparing to normal nodes and there is no longer need
in interfaces discover for nodes since a config value can be used
instead.
[1] https://review.opendev.org/c/x/whitebox-neutron-tempest-plugin/+/913238
Change-Id: Ida23fa855a7986ba5c5f9ca49663644715783a24
The use of retry decorator is easy and tempting,
many test methods may find various issues, therefore raising exceptions,
this change forces specific failure to be targeted by retry decorator
in order to avoid dismissing other exceptions which require a fix.
This is done by an obligatory regex, which validates
the exception caught and dismissed has the intended assertion message.
There is optional exception type argument, in order to let other
unexpected exceptions to be raised and fail the test.
Change-Id: Ib9a17d8c6c8778adf546c610dd070c14aff45ecd
Upstream network isn't as stable in downstream test execution,
test quickly checks traffic over time periods for configured limits,
asserts that there is no traffic loss to verify rate limitation
correctly.
This oftens fails on upstream gates, therefore adding the retry
decorator up to 3 times to get stable measures for test, while not
failing check/gate patches.
Change-Id: I7ad1fd74df4460554beb6b0ea89abcf8153645cb
This fixes tests in the following test files [1].
Method 'validate_command' is commonly used for openstack/oc commands,
therefore default ssh client should be the proxy host
(localhost on devstack, controller on podified for now).
Another new argument 'local_shell' allows to easily override and
execute any command locally (without ssh connection).
[1]
test_provider_network.py
test_router_flavors.py
test_security_group_logging.py
Change-Id: I9138b1081870da4162eeb40ca055e3547f5c12b9
Usually testing setups have openstack nodes implemented as
virtual machines running on the same hypervisor host.
For tests that perform power operations for the nodes it is
necessary to have an ability to run virsh commands from the
hypervisor host.
While there is a config parameter hypervisor_host that allows
to set the value, in some cases it is not possible to know what
will a hypervisor host name since it is selected randomally.
This patch adds code that as a fallback scenario tries to
determine hypervisor host name from ssh config file, that is
usually available on the proxy host (ansible controller host)
in case environment was deployed by ci-framework.
Change-Id: Icc71c5a0511f04c19ef9509a32278f4e70e92bd2
On podified environment it can be that more than one replica
of rabbit or galera is running on same node and some other
OCP node does not have such processes running while still being
OSP controller, i.e. host with OSP control plane services.
This patch assigns is_controller flag for OCP hosts according
to inventory data.
Change-Id: I5aa66350f6eca924c756f82b3c074136c3388f9e
After code was changed for capture traffic on ovn-controller-ovs
container rather than on OCP nodes bridges there is no longer need
to discover OCP nodes interfaces.
Change-Id: I7ef62a2e46758345034c61c06222f8845c6f71f8
The neutron pod or any other service contains separate custom
configuration buffer for non default configuration.
Any configuration change should retrieve and add existing configuration
first, this change adds such fix to avoid deleting previous settings.
This fix also waits for old service pod to be replaced before allowing
the test to continue.
Change-Id: I4028c9f9fed3d38c3bcaa698e3a119a9656b0886
Since some tests can be disruptive and can break some important
nodes an option for using only safe nodes was introduced.
With this option enabled, tests that are doing disruptive actions
will not use OSP controller nodes for this but will retry
to create objects for testing on a safer node. The option is
enabled by default.
In order to make test_l3ha_reboot_node to work properly,
discover_nodes() was changed to support nodes rescan when there
are shut off nodes.
test_l3ha_bring_down_interface was changed to use
node_tenant_interface defined in config file instead of letting
the test to discover it.
Changed default value for hypervisor_host to match the
current value used on environments deployed by ci-framework.
Moved all power operations functions to a separate class that
disruptive tests should inherit from.
Fixed incorrect calls of find_host_virsh_name by
adding required parameter and some other fixes in power
operations functions.
Change-Id: Iafe9b0210972e0c59529c4a09edbe11d68f91514
This patch adds a decorator for those functions that we want to
evaluate more than once in case of an assertion failure. Some
scenario tests are conditioned by the stability of the VM or
interfaces after their creation to obtain bandwidth measurements
according to QoS criteria. For these tests, this decorator allows
for multiple evaluations seeking stability in the measurements
Co-Authored-By: froyo@redhat.com
Change-Id: I4c562a97ff958e8c15408f3d06c54c24e9a2f1fb
For some reason, the list of CIDRs from an OCP node sometimes does not
include some of its IP addresses.
The method `get_ocp_main_ip` will use the hostname in order to identify
OCP nodes instead.
Change-Id: Ie3e4460d9d338aed21f62b4b17471aef26bebd4a
This patch enables shared parameter similarly to patch [0],
only on test class level.
For podified environments some tests will temporary enable 'shared'
parameter for external network. This is because it was decided that
on podified environments external network should not be shared
by default for some tempest api tests to work properly, see [1].
Podified environment should run all the tests that modify external
network one by one, in a single tempest thread.
[0] eacd16a847
[1] https://github.com/openstack-k8s-operators/ci-framework/pull/1862
Change-Id: Id841b5c1cb5b620d8fcf8852b6041b1f80257505
Since the plugin is recently used mainly for testing multi-node
podified environments it is worth to set this as default value
to avoid irrelevant failures and simplify podified job definitions.
Also, run_traffic_flow_tests parameter is now enabled by default.
Also, CONF.identity.uri_v3 is now preferred over CONF.identity.uri
Change-Id: Ib73f4fb0f8a465ba1e40137c0237211bb1642ef5
When the `ps` command is executed on a pod with `oc rsh <pod> ps ...`,
sometimes the exit code is not 0, but the command returns the expected
output.
This patch adds " || true" to that command in order to ignore the
returned code.
Related-issue: OSPRH-8352
Related-issue: https://github.com/kubevirt/kubevirt/issues/10240
Change-Id: I246adf9f10a2d08b64be001bf13bfb807fb62698
In case of podified setups without CRC, instead of using the IPs
for the OCP nodes from the inventory, their IPs should be obtained with
the command `oc get node/<nodename> -o yaml` from the status.addresses
field.
Change-Id: I9d5e614274adda61d57a177d8d9243993305564c
In case of podified setups, when the number of replicas is greater
than 1, the requests to the OVN databases should be sent to the
RAFT DB leader instance.
The requests can be sent from any OVN DB NB/SB pod, but the parameter
--db should be included in order to forward the requests to the leaders.
Change-Id: I802b9b685a7254b6eebae0bbe7d0033c06077349