24909 Commits

Author SHA1 Message Date
Rodolfo Alonso Hernandez
a660f77d38 Remove "mitogen" library installation from "docs" job
Since pyroute 0.6.2, this dependency is not needed anymore. The
pyroute version is bumped both in requirements and doc/requirements.

This patch bumps the pyroute2 version to 0.6.4 (newest version
provided in upper-requirements).

Change-Id: I627e902e32fe59c6829c3bc92af19abff4b70f9a
Related-Bug: #1928913
2021-06-07 12:39:28 +00:00
Rodolfo Alonso Hernandez
6afdff8494 Bump lower version of evenlet to 0.25.1
That fixes the lower-constraints job with pyroute >=0.6.2.

Change-Id: I849d777903b10a809947f0564205deb73e668c3c
Closes-Bug: #1931098
2021-06-07 12:38:27 +00:00
Zuul
c6ebae4395 Merge "Use payloads for ADDRESS_GROUP callback events" 2021-06-07 09:14:03 +00:00
Zuul
8f176a3887 Merge "Follow up for replacing assertItemsEqual" 2021-06-06 13:06:40 +00:00
Slawek Kaplonski
1e8197fee5 Add tests for service_type API's new policy rules
Related-blueprint: bp/secure-rbac-roles
Change-Id: I0cbe260fc786d00b014c804ae74d99c8f9685e88
2021-06-05 22:48:33 +00:00
Slawek Kaplonski
962d2539a1 Change API policy for service_type to be available for all readers
Service type resource don't have "project_id" so using "PROJECT_*"
rules wasn't working fine.
And this resource should be available for all users so this patch
switches its check_str to be "role:reader" which works for all
types of SYSTEM and PROJECT scope users.

Related-blueprint: bp/secure-rbac-roles
Change-Id: If28e70252c1f9ec76502699fad2d5a2aece8f4fb
2021-06-05 22:48:02 +00:00
Zuul
5d33a6b9dd Merge "[Fullstack] Add segmentation_id update test" 2021-06-05 09:28:15 +00:00
Zuul
0fb97c909e Merge "Improve Subnet create performance" 2021-06-05 04:06:07 +00:00
Zuul
28e08c053b Merge "Make phynet paramter also is optional when network_segment_range enabled" 2021-06-05 01:03:45 +00:00
Zuul
f7bac83f85 Merge "Improve Subnet delete performance" 2021-06-05 00:57:35 +00:00
Zuul
bb85e7a2e4 Merge "Skip "test_keepalived_spawns_conflicting_*" tests" 2021-06-04 23:26:31 +00:00
Zuul
67c036e611 Merge "Set the default `Backend.lookup` timeout to 2 seconds" 2021-06-04 23:20:11 +00:00
Zuul
afcaf6805d Merge "New Quota driver `DbQuotaNoLockDriver`" 2021-06-04 23:19:37 +00:00
Rodolfo Alonso Hernandez
074c131b57 Disable pep8 import check for pyroute2 library
Since version 0.6.2, pyroute2 library dynamically imports the needed
modules when loaded. A static analysis will fail when checking the
import references.

Change-Id: I5aaf9494a2d5c2533199e6b92d4df8fe785f83a3
Closes-Bug: #1930750
2021-06-03 20:06:44 +00:00
Nurmatov Mamatisa
17adb4c3fb Use payloads for ADDRESS_GROUP callback events
This patch switches over to payload style of callbacks for the remaining
ADDRESS_GROUP based callback events.

Change-Id: I7aafd00211d66b0d1cc3bbdc79efc344c2acc700
2021-06-03 18:40:14 +00:00
Takashi Natsume
05ee23ea15 Follow up for replacing assertItemsEqual
The assertItemsEqual has been replaced with assertCountEqual
since I7c20fec08e5dc9f67b34100c925ea6724bbd25f0 .
However the hacking check has not been enabled.

This patch enables the hacking check
and replace remaining assertItemsEqual with assertCountEqual.

Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Change-Id: I86202f36f1b7c0bce8b29558ad1e377c336b7258
2021-06-03 16:04:07 +00:00
Zuul
7e98d18927 Merge "Switch neutron-rally-task job to ML2/OVS and skip "test_models_sync"" 2021-06-03 15:35:11 +00:00
Zuul
63d8788d0f Merge "Change minversion of tox to 3.18.0" 2021-06-03 12:40:59 +00:00
Slawek Kaplonski
4de350a54a Switch neutron-rally-task job to ML2/OVS and skip "test_models_sync"
After switch of the default Neutron backend to the ML2/OVN
in the Devstack we had 2 rally jobs running actually with the
same Neutron backend.

This patch configures explicitly ML2/OVS as a backend in the
'neutron-rally-task' job so it is like it was before the change in
the Devstack repo.

In order to fix the CI, this patch also skips "test_models_sync" test
cases. Once [1] is fixed and released, we'll be able to create SQL
tables specifying the "_constraints_included" flag. When creating a
table, this flag should be set to "False" in order to be able to set
the "index" and "unique" parameters for each column in the table
definition, as we do now.

"test_update_router_admin_state" is excluded from
"neutron-ovn-tempest-slow" until LP#1890445 is fixed.

Set "test_restart_wsgi_on_sighup_multiple_workers" as unstable.

CI job "neutron-ovn-tempest-slow" has been set to non-voting until
LP#1930402 is fixed.

Related-Bug: #1929518
Related-Bug: #1890445
Related-Bug: #1930402
Related-Bug: #1930367

[1]https://github.com/sqlalchemy/alembic/issues/848

Change-Id: I38ed7931169434fecbbefa3762f51a522ef3c509
2021-06-02 17:19:54 +00:00
Zuul
0bdf3b56e0 Merge "Use local and ip address to create vxlan interface" 2021-06-02 10:38:57 +00:00
Anthony Timmins
4cd11f4dee Use local and ip address to create vxlan interface
Attempting to terminate a vxlan on the lo interface with
multiple ip addresses fails. This seems to be because only
the first ip address on the interface is used. If this address
is invalid for vxlan creation (ie. 127.0.0.1), the vxlan
interface will be created, but will not have a source ip
address, and will therefore be non-functional. To remedy this
issue, when L2population is used, we can set the local
argument to the local_ip, thus ensuring the intended ip
address is configured.

Closes-Bug: 1929998
Change-Id: I9c54a268fc4ef9705637556ecba161bd6523a047
Signed-off-by: Anthony Timmins <atimmins@datto.com>
2021-06-01 11:02:21 -04:00
Zuul
c8a4beb95b Merge "[Doc] Update Freenode to OFTC as our IRC server" 2021-06-01 06:49:59 +00:00
Slawek Kaplonski
c4fb1d1711 [Doc] Update Freenode to OFTC as our IRC server
Change-Id: I4e4d819e271495fa650a275246df8925ef84fd25
2021-05-31 21:54:08 +02:00
Rodolfo Alonso Hernandez
b141aed512 Remove leading zeroes from an IP address
In python3.9, ipaddress.IPv4Interface does not support defining an
IP address with leading zeroes. E.g.:

>>> ipaddress.IPv4Interface('00.0.0.0/0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.9/ipaddress.py", line 1390, in __init__
    IPv4Address.__init__(self, addr)
  File "/usr/lib/python3.9/ipaddress.py", line 1307, in __init__
    self._ip = self._ip_int_from_string(addr_str)
  File "/usr/lib/python3.9/ipaddress.py", line 1199, in
_ip_int_from_string
    raise AddressValueError("%s in %r" % (exc, ip_str)) from None
ipaddress.AddressValueError: Leading zeros are not permitted in '00' in
'00.0.0.0'

Change-Id: I29137b22c158e979e81be9e41464948aa99f5498
Closes-Bug: #1930222
2021-05-31 11:41:04 +00:00
Zuul
5a6f61af4a Merge "Add tests for metering API's new policy rules" 2021-05-27 17:28:40 +00:00
Zuul
43e0fb05b4 Merge "Add tests for port API's new policy rules" 2021-05-27 17:28:16 +00:00
Zuul
6fb4de7243 Merge "Update Neutron's Liuetenants" 2021-05-26 17:53:56 +00:00
Zuul
dd24662e53 Merge "Add tests for FIPs API's new policy rules" 2021-05-25 10:34:07 +00:00
Zuul
f7302a0cfe Merge "Add tests for router API's new policy rules" 2021-05-25 10:25:21 +00:00
Zuul
4afe223332 Merge "Add tests for segments API's new policy rules" 2021-05-25 10:24:58 +00:00
Zuul
19453cf655 Merge "Add tests for subnetpool API's new policy rules" 2021-05-25 10:24:40 +00:00
Slawek Kaplonski
f83c77bb12 Add tests for port API's new policy rules
Related-blueprint: bp/secure-rbac-roles
Change-Id: I7fa382b0862bd386b480d836a6b466c27b46677c
2021-05-25 10:49:18 +02:00
LIU Yulong
444ef49bea Add test cases for concurrently Object delete
Verify if concurrently delete neutron DB records will
encounter the StaleDataError which suggest us to add
"confirm_deleted_rows=False" to mapper configuration.

Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/777581

Change-Id: Ia8935d5dd87402bedfd7aa9df9dfcb0ce06f8e39
Related-Bug: #1916889
2021-05-24 07:11:58 +00:00
Zuul
b24d2fd769 Merge "Deprecate and remove "get_agents_db"" 2021-05-23 21:33:59 +00:00
Zuul
c6c4d85a18 Merge "HA-non-DVR router don't need manually add static route" 2021-05-22 05:05:09 +00:00
Oleg Bondarev
f52280287f Improve Subnet create performance
- pass network dict from ml2 plugin to _create_subnet_postcommit
- skip ipam subnet fetch for non ipv6 auto-address subnets
- don't count subnets (DB request) if subnet has no segment

Change-Id: Iaecfda2700c5316cb25a93496d24ece366e40a4a
2021-05-21 15:03:52 +00:00
Oleg Bondarev
bdd50ffcde Improve Subnet delete performance
- don't re-fetch subnet object from DB
- network is not used in SubnetContext when deleting subnet

Above gives ~35% improvement

Change-Id: I34f850782092f771482a297ae1e68a63ffb027c1
2021-05-21 15:03:38 +00:00
Zuul
4882fa34bb Merge "[DVR] Send allowed address pairs info to the L3 agents" 2021-05-21 09:50:17 +00:00
Takashi Natsume
ab0895fdf4 Change minversion of tox to 3.18.0
The patch bumps min version of tox to 3.18.0 in order to
replace whitelist_externals with allowlist_externals option.
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Change-Id: If304d7c277850470448ca9bcd0f0bfd250ec001d
2021-05-21 08:06:40 +00:00
Zuul
2980a962a0 Merge "Add mitogen to the docs requirements" 2021-05-20 15:50:15 +00:00
Zuul
798fd70b91 Merge "[OVN] Fix: Disabling snat after it was enabled" 2021-05-20 15:40:37 +00:00
Slawek Kaplonski
a75fc891bc Add mitogen to the docs requirements
Mitogen is dependency of the pyroute2 and was added link to it
the docs in pyroute2 0.6.1. Due to the missing mitogen in our
docs venv, docs job was failing.
So let's install mitogen as docs env requirement, at least for now
to unblock our gate.
Later we can think about better solution as e.g. adding it properly to
the requirements or report and fix the issue in the pyroute2.

Change-Id: Iba37581916842bb5fc9f42f7f483d3431747bf8d
Closes-Bug: #1928913
2021-05-20 09:00:40 -04:00
Zuul
5307cd8abe Merge "Add tests for QoS API's new policy rules" 2021-05-20 11:12:46 +00:00
Zuul
f3d1b8b9c2 Merge "Add tests for Security Groups API's new policy rules" 2021-05-20 09:22:03 +00:00
Zuul
b894d0103d Merge "Add tests for L3 conntrack helper API's new policy rules" 2021-05-20 08:49:15 +00:00
Rodolfo Alonso Hernandez
951a3f0eb3 Deprecate and remove "get_agents_db"
This method is superseded by "get_agent_objects".

Projects using this method:
- networking-odl:https://review.opendev.org/c/openstack/networking-odl/+/792014

Trivial-Fix

Change-Id: I247bf2885a8b1bf9205f25d45ee3b905b56b0c05
2021-05-20 08:04:47 +00:00
Rodolfo Alonso Hernandez
54420d04dd Skip "test_keepalived_spawns_conflicting_*" tests
SIGHUP signal command to stop those processes does not return and
the test (and eventually the whole testsuit) timeouts.

Until a proper fix is implemented, these tests will be skipped
temporarily.

Change-Id: I592d8b7bb2afe5cd93cbb0d0ea7062f1b724ff2a
Related-Bug: #1921154
2021-05-20 07:56:29 +00:00
Rodolfo Alonso Hernandez
e135a8221d New Quota driver `DbQuotaNoLockDriver`
This new quota driver, ``DbQuotaNoLockDriver``, does not create a lock
per (resource, project_id) but retrieves the instant (resource,
project_id) usage and the current (resource, project_id) reservations.
If the requested number of resources fit the available quota, a new
``Reservation`` register is created with the amount of units requested.

All those operations are done inside a DB transaction context. That
means the amount of resources and reservations is guaranteed inside
this transaction (depending on the DB backend isolation level defined)
and the new reservation created will not clash with other DB transation.
That will guarantee the number of resources and instant reservations
never exceed the quota limits defined for this (resource, project_id).

NOTES:
- This change tries to be as unobtrusive as possible. The new driver
  uses the same ``DbQuotaDriver`` dabatase tables (except for
  ``QuotaUsage``) and the same Quota engine API, located in
  ``neutron.quota``. However, the Quota engine resources implements some
  particular API actions like "dirty", that are not used in the new
  driver.
- The Pecan Quota enforcement hooks,
  ``neutron.pecan_wgsi.hooks.quota_enforcement``, execute actions like
  "resync", "mark_resources_dirty" or "set_resources_dirty", that has
  no meaning in the new driver.
- The isolation between the Quota engine and the Pecan hook, and the
  driver itself is not clearly defined. A refactor of the Quota engine,
  Quota service, Quota drivers and a common API between the driver and
  the engine is needed.
- If ``DbQuotaDriver`` is deprecated, ``CountableResource`` and
  ``TrackedResource`` will be joined in a single class. This resource
  class will have a count method (countable) or a hard dependency on a
  database table (tracked resource). The only difference will be the
  "count" method implementation.

Closes-Bug: #1926787

Change-Id: I4f98c6fcd781459fd7150aff426d19c7fdfa98c1
2021-05-20 07:55:59 +00:00
Rodolfo Alonso Hernandez
9e6b7a2284 Install "pyroute2" as a doc job depedency
By adding this library to the doc requirements file, tox will install
it limiting the upper version according to
"requirements/upper-constraints.txt".

"pyroute2" is called in tox doc job as mentioned in [1]. Since version
0.6.0, the library structure changed. To keep backwards compatibility,
all modules are imported from the main namespace. As reported in the
referred bug, the library "mitogen" must be installed first.

In stable releases that use older "pyroute2" versions, this library is
not needed but this patch should enforce the upper library version.

Related-Bug: #1928913

[1]2a4074918c/doc/source/conf.py (L33-L40)

Change-Id: I0cf458f4532d313e593a943653e8310b93ac1e9c
2021-05-19 14:31:31 +00:00
Rodolfo Alonso Hernandez
2a4074918c Remove call to "fixup_ubuntu"
Since [1] this method is not in Devstack repository. Bionic is no
longer supported.

[1]https://review.opendev.org/c/openstack/devstack/+/788754

Change-Id: Ied3442289864656654d3745d3ea7841a5aae713c
Closes-Bug: #1928805
2021-05-18 14:13:31 +00:00