Commit Graph

2840 Commits (master)

Author SHA1 Message Date
Zuul fde58ffa30 Merge "Improve logs for test_alive_agents_are_consistent_along_time" 24 hours ago
Eduardo Olivares 654a7257bf Improve logs for test_alive_agents_are_consistent_along_time
When this test fails or when retries are needed, it is not clear why
it failed. The logs say "The list of agents has changed".
It should be clear how that list changed.

Change-Id: I0e6a070dcf5808b0ab4379cb41e74e5ff352856a
2 days ago
Eduardo Olivares e3231d1384 Use proper ceritificate when TLS-e is configured
When the Openstack auth-url is based on https, using the openstackclient
requires to provide the path to the certificate file from the

This patch also fixes the octavia deploy_ipv4_lb method: it tries to
find the external IPv4 subnet using its name. With this patch, the
external network is found (only one external network is supported so
far) and then its IPv4 subnet is returned

Change-Id: I58f7aae796478eda0bff87ec60b62c940a67e677
1 week ago
Eduardo Olivares da0d344f5f Add retries to find created ports at test_ovsdb_transation
This test may be unstable because the ports are created and immediately
they are obtained from the nova API.
When a port creation request succeeds, it doesn't mean it has already
been created, but the POST request is being processed (response 201). It
may take a while to really create the port

Change-Id: Iea69f2252dc766ffcc0513331535ceebe9f80ddc
1 week ago
Slawek Kaplonski fb96fe49bb Add readthedocs.yaml configuration file
This file is used to configure RTD docs builds for the project.

Change-Id: I802dae3f9dae8178c4a152339125d9f71806d73f
2 weeks ago
Slawek Kaplonski f45e83a175 Add upper-constraints file to the RTD requirements file
It's needed to cap our dependencies also on the RTD when documentation
is build to not use newer versions of packages then should be used.

For example now documentation builds on the RTD was failing because of
urllib3 2.0 which isn't compatible with Tobiko (tobiko.http._session
module in particular).
Module urllib3 is capped to 1.26.8 version in our upper-constraints.txt
file but that wasn't used in RTD builds until now.

Change-Id: I279767393716371dd16afe801c66814acd364171
2 weeks ago
Eduardo Olivares ad44b6cb68 Update latest stable tag to 0.6.10
Change-Id: I0767745991ec620dcfad6d8ac2c3767f1b39b7dc
2 weeks ago
Omer c3cb8ce38c Remove Heat stacks from Octavia
Heat might not be supported in OSP18.

After the Tobiko team decided to remove the dependency on Heat, this
patch removes Heat stacks and templates from Octavia.

Note: Backend servers (such as Ubuntu) and subnet still use heat
templates, and Octavia uses them, but after this patch there
won't be any Octavia API calls which will be called by Heat.

Change-Id: I5c335d68c093e8e5e1d247c300186859b889bd39
2 weeks ago
Roman Safronov 19aa88b433 Skip test_ovs_interfaces_are_absent after ovs2ovn
After OVN migration some OVS-specific interfaces can still present
on nodes until VMs are migrated or nodes rebooted.
Skipping this test after OVN migration will help to avoid
irrelevant failures.

Change-Id: I50f91477a063d8f3feefb26c70e4ea041a6d13f4
2 weeks ago
Zuul 538922edec Merge "Fix Octavia validators error about missing member" 2 weeks ago
Zuul c48c6368f0 Merge "[BGP] Port tests without FIP" 2 weeks ago
Zuul 98d70c4288 Merge "Add faults ovn_bgp_agent and frr tests" 2 weeks ago
Zuul a260031555 Merge "Allow custom ubuntu timeouts" 3 weeks ago
Omer 743adacdf4 Fix Octavia validators error about missing member
So far, some validations in Octavia might have run too early in the
validation flow, for example, a request is being sent to a member which
is still not ready.

This patch replaces the simple assertEqual, with an exception which will
be raised (RoundRobinException), in case that one member is not ready

The validator method caller wraps the call with a try/except clause, to
make sure Octavia has enough time to launch a ready-vm.

Change-Id: Ie3821441ff1659b7f919a34e517c32b552eaf544
3 weeks ago
Zuul 761de81433 Merge "Improve Octavia ovn provider scenario test" 3 weeks ago
Zuul 1ea31ce8f9 Merge "Fix error message when a stack is not deleted" 3 weeks ago
Zuul c71a0c69f2 Merge "Make test_z99_reboot_controller_galera_main_vip more stable" 3 weeks ago
Zuul 4a33692f55 Merge "Skip some more tests related to ovn-migration" 3 weeks ago
Omer 52cbda2b4c Improve Octavia ovn provider scenario test
So far Octavia ovn-provider did not allow the end user to curl the LB

Now that it is allowed, this patch improves the Octavia ovn-provider
scenario test by creating the Octavia resources using ports 80/8080 and
cURLing them (instead of only SSHing them to generate traffic).

Change-Id: I32670197f4600f09d07b36810fd081f1512b77bc
3 weeks ago
Roman Safronov 6bc9c6eff7 Allow custom ubuntu timeouts
Default timeouts of 900 and 1500 seconds which are suitable for
upstream CI are too long for downstream CI causing redundant
15 minutes retries while 2-3 minutes are enough to make a decision
that something went wrong. This patch allows
configuring custom values for ubuntu vm connectivity timeouts.

Change-Id: If33991b0b932d7615f7072a4c2121fde4d5bcdc8
3 weeks ago
Eduardo Olivares 38e7f70acc Remove Fedora, Centos and RHEL from tobiko tests
The scenario tests were not really executed because they were marked as
The benefit from using this images in some functional tests is irrelevant
compared with the impact in job duration and stability.
Due to these reasons, it has been decided to use only cirros and ubuntu
(for the test that required advanced images).

Change-Id: I168b1f071b3a77eb0b24586fd403d12165db1db2
3 weeks ago
Eduardo Olivares 1408422d72 Fix error message when a stack is not deleted
When a stack is not deleted and the retry loop ends, tobiko raises an
exception with the following error message:
HeatStackDeletionFailed: stack <name> status 'DELETE_COMPLETE' not in

This is a misleading message fixed by this patch, because the real
problem is that the stack remains in DELETE_COMPLETE status and was not
properly deleted

Change-Id: I661e7e727e60dc16195e6589fac47aa7ecb16f0c
3 weeks ago
Eduardo Olivares e0202499ac Make test_z99_reboot_controller_galera_main_vip more stable
This test was sometimes failing during the verification of the number of
ports obtained after the controller reboot.
Two issues have been identified and corrected:
- The information obtained from nova/heat for some VMs showed no ports
  and in some cases a port had been created for those VMs. The test is
  modified to try to obtain the port from neutron in those cases.
- The final list of ports after the reboot was obtained before
  the loop checking VM ports ends. It has been moved to obtain the list
  after that loop, which looks more stable.

Change-Id: If57c2fc9451a89813f63d6c84e5ad86b991cfd3f
4 weeks ago
Eduardo Olivares b5b86ee56b Skip some more tests related to ovn-migration
Added option to skip flaky tests during ovn-migration testenvs.
Added decorator to StatelessSecurityGroupTest in order to skip these
tests during ovn-migration too.

Change-Id: I2d30df19dc3e7102b430254db1585e6df33702df
1 month ago
Eduardo Olivares 0962d8e266 Update latest stable tag to 0.6.9
Change-Id: I4380d20a06fe7f2e0a9035782601a78143e8db52
1 month ago
Eduardo Olivares 3d0a48cd8e [BGP] Port tests without FIP
Added NoFipPortTest class to test_port module. They are only executed
when BGP is configured and expose_tenant_networks is enabled. These
tests use a VM without FIP. The tests connect to the fixed IPs from its
tenant networks.

Change-Id: I458ab8f8ebe92a0f53bbd10675ae161450ffc0ff
1 month ago
Eduardo Olivares a3116bf0e1 Add faults ovn_bgp_agent and frr tests
This patch adds two faults tests. They restart the ovn_bgp_agent and
the frr services, respectively, on all the overcloud nodes where these
services run.
When BGP is configured with expose_tenant_networks enabled, faults tests
will create an extra VM without FIP to verify connectivity to its tenant
IP addresses.

Change-Id: I118427b41e1c97d075572a5ebee21025ac5cc967
1 month ago
Zuul bc07bea7b4 Merge "Add new tox env and workflow for ovn_migration sanity tests" 1 month ago
Zuul e6b2d4813a Merge "Don't run some of the stateless SG tests in ovn migration workflow" 1 month ago
Slawek Kaplonski d5bc5c7617 Add new tox env and workflow for ovn_migration sanity tests
This patch adds new tox env "ovn_migration_sanity" which runs only tests
marked as "ovn_migration" and explicitly skip all tests marked as

This patch additionally modifies check-resources-ovn-migration workflow
so it will now first check resources after migration (run
"ovn_migration" tox env) and next run ovn_migration sanity tests too.

Change-Id: Ib50bfb0c294ee305fa7bee53c048939479e2e436
1 month ago
Slawek Kaplonski 3aa8b57abf Don't run some of the stateless SG tests in ovn migration workflow
This commit adds new pytest mark "skip_during_ovn_migration" to mark
some tests to be always skipped during migration from ML2/OVS to
ML2/OVN and marks as such tests from the
StatelessSecurityGroupInstanceTest class.

The reason why those tests shouldn't be run during the ovs to ovn
migration is that we are skipping them in ML2/OVS env so resources
required by those tests aren't created and later, when env is ML2/OVN
already we can't really verify them as we don't have required resources

This patch modifies also ovn_migration tox env so it will run all
neutron scenario tests EXCEPT tests marked with this new pytest mark as

Change-Id: Iaa1e8b0d325ea690c6d6cb89e06006e7bbed9237
1 month ago
Eduardo Olivares 37ad5aa3b3 Unskip UbuntuExternalPortTest tests on upstream jobs
Decorator skip_unless_osp_version should not skip tests when the osp
version is not known. Those tests were wrongly being skipped in all
upstream jobs

Change-Id: I445fc3a04c70d46fc4fa32aadc88068f9de70533
1 month ago
Eduardo Olivares 556f60bfc2 Workaround to avoid connectivity problems from ubuntu VM instances
With some recent ubuntu release, routes to nameservers are added to VLAN
interfaces. This breaks some tests when they are executed in downstream
jobs because the affected VM instances are unable to reach the external

Change-Id: Ica2d029792b543222193cfd4175edc15f0b981ab
1 month ago
Eduardo Olivares 109802be5f Create only one Sec Group per fixture
This patch includes some changes wrt StatelessSecurityGroupFixture:
- a task that cleans up the security groups generated with this fixture
  is added to the tobiko-cleanup role
- when TOBIKO_PREVENT_CREATE env variable is set, the security groups
  are not created (they should have been created before)
- the lockutils.synchronized decorator is used to avoid that two
  security groups with the same name are created concurrently
- neutron.get_security_group( did not work because that method
  does not work with the security group name; it is replaced by
  neutron.list_security_groups( instead

Change-Id: I3ab84e138049a27db017630b32d81d290aa5f6cd
2 months ago
Zuul d95d75eae7 Merge "Use Subnet pools to avoid cidr concurrency issues" 2 months ago
Zuul 847db18c6d Merge "Refactor test_alive_agents_are_consistent_along_time" 2 months ago
Eduardo Olivares a6e7a522df Use Subnet pools to avoid cidr concurrency issues
This patch removes the tobiko.openstack.neutron._cidr module that was
used to generate CIDRs whenever tobiko was going to create a new subnet.
That method had problems when several tobiko workers running in parallel
tried to create subnets. Collisions happened sometimes, resulting in the
creation of two subnets with a common CIDR that could fail if they were
connected to a common router.

This patch uses the neutron subnet pool feature [1]. When Tobiko creates
a subnet, it will not calculate a free IPv4/IPv6 CIDR, but will
associate that subnet to common Subnet Pools, shared by the different
tests. When the subnet is created, it is neutron responsibiltiy to
allocate a free CIDR for that subnet, based on the created Subnet Pools.

The method that creates the Subnet Pools uses an
@oslo_concurrency.lockutils.synchronized decorator to guarantee that two
Tobiko workers do not create the same Subnet Pool at the same time.


Change-Id: I3b09e468310e06f8c4b7d7dbec02e4b2e6f67530
2 months ago
Eduardo Olivares 75939feb20 Fix faults tests when External LB is configured
Tobiko faults tests that depend on the Main VIP resource configured
should be skipped because this resource is not created when an External
LB is used.
Disabling the haproxy resource should be skipped because that resource
does not exist then an External LB is used.

Change-Id: Ic20fa2f321989aa4c49aeff8b4e01b2025bd4160
2 months ago
Eduardo Olivares 2ad24dd2f0 Do not upgrade virtualenv to the latest version
virtualenv is a requirement for tox, so it is installed when tobiko
roles install tox.
Besides, virtualenv is also defined in the openstack requirements
upper-constraints.txt file, specifying which one is the virtualenv
valid upper version.
Due to this, virtualenv is removed from the task that upgrades other
packages as part of the tobiko-ensure-python3 role.

Change-Id: I41b76d7041face19635c9fd35e773b9144d25c30
2 months ago
Eduardo Olivares eee8a30d97 Refactor test_alive_agents_are_consistent_along_time
With this patch, the method is based on two tobiko.retry loops.
The outer loop is used to retry in case of any unexpected result when
neutron.list_agents is called: neutron exception, some dead agents or
the list of agents does not match the previously obtained list. The
retry timeout will be 180 seconds by default.
The inner loop is used to count the number of consistent results, i.e.
how many times the obtained list of neutron agents is identical. The
test passes if the result is consistent for 5 times by default.

Change-Id: I5891bdda50e604de45ce55aaa86440988c55cca0
2 months ago
Zuul f8b0765ecc Merge "Catch some exceptions when novaclient.list_services is run" 2 months ago
Eduardo Olivares cfd7d864d4 Fix neutron requests that are sent by background processes
Test test_ovsdb_transation sends several neutron port creation requests
in parallel, using background processes. This failed when using SSL, an

In order to avoid that exception, this patch makes the background
processes send their neutron requests using different keystone

Change-Id: I91739e40fe4e27eb0280702f697bea36eb6685db
2 months ago
Zuul 69bb4e65ae Merge "Add designate upstream job" 2 months ago
Omer 5f5bc07288 Add designate upstream job
This patch tries out the upstream designate job we have in tobiko.

Change-Id: Ide94ca0118b77c2c9a1fda1331c898272a5fce9b
2 months ago
Eduardo Olivares 25450dcd64 Fix captured exceptions and add some logs to test_ovsdb_transation
Change-Id: I6f659381d892f4a321d0cd7d137f1fd756d85066
2 months ago
Eduardo Olivares adea7754ff Catch some exceptions when novaclient.list_services is run
Tobiko faults tests check the list of nova services after the test
performs some disruption on the cloud. The list_services request could
fail after the disruption while the cloud is recovering and the test
should not fail due to this, but retry some times until the cloud is
healthy again.

Change-Id: I540cf2c39ef0d6cbc3cd53e917b7652ddfea5a57
2 months ago
Zuul 8ea76d1253 Merge "Update github host key" 2 months ago
Zuul 9d0218df4c Merge "Add openstacksdk config options method" 2 months ago
Eduardo Olivares 31e3e4719d Update github host key
Change-Id: I4c5ee7059240ce2c7fdbb41d6ec03afc3debe15c
2 months ago
Zuul 34c88ebbb8 Merge "Remove designate_faults testenv" 2 months ago