neutron/neutron/db
Felix Huettner d25c129ec2 Reduce lock contention on subnets
HINT: This isn't a clean backport, as we keep the subnet in-use field.
We can't backport the db update that would remove the field.

in [1] a lock was introduced with the goal of preventing subnets from
being deleted while ports are being created in them in parallel.
This was acheived by aquiring an exclusive lock on the row of the
subnet in the Subnet table when adding/modifying a port or deleting
the subnet.

However as this was a exclusive lock it also prevented concurrent port
modifications on the same subnet from happening. This can cause
performance issues on environment with large shared subnets (e.g. a
large external subnet).

To reduce the lock contention for this case we split the lock in two
parts:

* For normal port operations we will aquire a shared lock on the
  row of the subnet. This allows multiple such operations to happen in
  parallel.
* For deleting a subnet we will aquire an exclusive lock on the row of
  the subnet. This lock can not be aquired when there is any shared
  lock currently on the row.

With this we maintain the same locking level as before, but reduce the
amount of lock contention happening and thereby improve throughput.

The performance improvement can be measured using rally test [2].
(improving from 21 to 18 seconds).
Alternatively it can be tested using 250 parallel curl calls to create a
port in the same network. This improves from 113s to 42s.

[1]: https://review.opendev.org/c/openstack/neutron/+/713045
[2]: https://github.com/openstack/rally-openstack/blob/master/samples/tasks/scenarios/neutron/create-and-delete-ports.json

Closes-Bug: #2009055
Change-Id: I31b1a9c2f986f59fee0da265acebbd88d2f8e4f8
(cherry picked from commit c0af5b3b5e)
2023-09-27 11:14:54 +02:00
..
allowed_address_pairs Clean up deprecated sqla model import paths 2017-02-25 03:42:34 +00:00
availability_zone Always create a "router_extra_attributes" register per router 2022-11-15 22:50:32 +01:00
extra_dhcp_opt Cleanup _find_related_obj 2017-02-25 06:29:33 +00:00
metering Replace "tenant_id" with "project_id" in metering service 2021-10-28 09:42:52 +00:00
migration Make DB migration creating indexes in RBACs conditional 2023-06-09 09:12:56 +00:00
models Use HasStandardAttributes as parent class for Tags DB model 2023-09-15 14:08:00 +00:00
network_dhcp_agent_binding Improve scheduling L3/DHCP agents, missing lower binding indexes 2023-02-13 17:53:00 +01:00
port_security Cleanup _find_related_obj 2017-02-25 06:29:33 +00:00
qos Change RBAC relationship loading method to "joined" 2023-05-30 12:27:22 +02:00
quota Implement specific tracked resource count method per quota driver 2022-07-28 06:01:18 +02:00
__init__.py
_utils.py Finish the new DB engine facade migration 2020-11-24 09:20:35 +00:00
address_group_db.py Unify the states format for address group payloads 2021-10-26 19:16:46 -05:00
address_scope_db.py Allow sharing of address scopes via RBAC mechanism 2020-03-19 16:51:39 +02:00
agents_db.py Add the corresponding DB context to all SQL transactions 2022-04-08 09:09:54 +00:00
agentschedulers_db.py Handle properly ObjectNotFound while deleting network from DHCP agent 2022-05-02 12:27:08 +02:00
allowedaddresspairs_db.py [DVR] Send allowed address pairs info to the L3 agents 2021-05-18 12:58:23 +02:00
data_plane_status_db.py Fix pep8 E128 warnings in non-test code 2019-03-12 21:22:33 +00:00
db_base_plugin_common.py Merge "Add the corresponding DB context to all SQL transactions" 2022-04-08 13:08:32 +00:00
db_base_plugin_v2.py Reduce lock contention on subnets 2023-09-27 11:14:54 +02:00
dns_db.py Fix pep8 E128 warnings in non-test code 2019-03-12 21:22:33 +00:00
dvr_mac_db.py [sqlalchemy-20] Add missing DB contexts in L3 methods 2022-05-15 11:39:54 +00:00
external_net_db.py Add the corresponding DB context to all SQL transactions 2022-04-08 09:09:54 +00:00
extradhcpopt_db.py Utilize bulk port creation ops in ml2 plugin 2019-03-21 11:31:36 +00:00
extraroute_db.py Check router routes connectivity when GW port is updated 2021-06-24 11:54:17 +00:00
flavors_db.py delete common_db_mixin 2019-04-17 11:25:41 -06:00
ipam_backend_mixin.py Reduce lock contention on subnets 2023-09-27 11:14:54 +02:00
ipam_pluggable_backend.py Allow to pass EUI64 IP address as fixed ip for the port 2022-10-05 07:11:25 +00:00
l3_agentschedulers_db.py Improve scheduling L3/DHCP agents, missing lower binding indexes 2023-02-13 17:53:00 +01:00
l3_attrs_db.py Always create a "router_extra_attributes" register per router 2022-11-15 22:50:32 +01:00
l3_db.py Don't allow deletion of the router ports without IP addresses 2023-07-04 14:48:34 +00:00
l3_dvr_db.py [OVN] Prevent Trunk creation/deletion with parent port bound 2023-07-03 22:26:35 +00:00
l3_dvr_ha_scheduler_db.py Fix connection between 2 dvr routers 2018-11-01 18:01:25 +01:00
l3_dvrscheduler_db.py Accept a port deletion with missing port binding information 2022-09-30 03:31:59 +02:00
l3_fip_pools_db.py Revert "Set system_scope='all' in elevated context" 2021-06-15 10:29:20 +02:00
l3_fip_port_details.py remove the neutron.db._resource_extend module 2019-01-02 13:02:06 -07:00
l3_fip_qos.py [QoS][L3] Use floating IP OVO QoS fields 2022-03-15 09:14:24 +00:00
l3_gateway_ip_qos.py Fix remaining typos in comments and tests 2022-07-06 21:20:27 +05:30
l3_gwmode_db.py Improve Router callback system's publish events 2021-12-05 04:36:36 +00:00
l3_hamode_db.py Prevent router_ha_interface port from being removed via API 2023-02-28 14:32:50 +00:00
l3_hascheduler_db.py use payloads for PORT AFTER_UPDATE events 2021-07-07 21:01:24 +00:00
local_ip_db.py Add Local IP Extension and DB 2021-11-11 10:08:23 +03:00
models_v2.py Reduce lock contention on subnets 2023-09-27 11:14:54 +02:00
network_ip_availability_db.py Add the corresponding DB context to all SQL transactions 2022-04-08 09:09:54 +00:00
ovn_hash_ring_db.py [OVN] Cleanup old Hash Ring node entries 2023-08-30 11:31:55 +01:00
ovn_revision_numbers_db.py Refactor the OVN revision module to access the DB correctly 2022-05-15 01:38:25 +00:00
port_device_profile_db.py Add port device profile extension 2021-01-22 16:17:30 +00:00
port_numa_affinity_policy_db.py Add port NUMA affinity policy 2020-11-13 15:49:34 +00:00
portsecurity_db.py remove the neutron.db._resource_extend module 2019-01-02 13:02:06 -07:00
portsecurity_db_common.py Start enforcing E125 flake8 directive 2019-07-19 23:39:41 -04:00
provisioning_blocks.py Use neutron-lib method is_session_active 2022-08-11 05:58:44 +02:00
rbac_db_mixin.py Replace "target_tenant" with "target_project" in RBAC OVOs and models 2021-12-03 10:48:57 +00:00
rbac_db_models.py Replace "target_tenant" with "target_project" in RBAC OVOs and models 2021-12-03 10:48:57 +00:00
securitygroups_db.py Revert "Delete sg rule which remote is the deleted sg" 2023-05-24 02:49:59 +02:00
securitygroups_rpc_base.py Add the corresponding DB context to all SQL transactions 2022-04-08 09:09:54 +00:00
segments_db.py use payloads for SEGMENT events 2021-05-05 15:17:50 +03:00
servicetype_db.py Switch to new engine facade for ExtraRoute_dbonly_mixin 2019-06-28 10:30:56 +00:00
standardattrdescription_db.py Use neutron-lib standard_attr 2021-09-15 09:19:26 +00:00
subnet_service_type_mixin.py remove the neutron.db._resource_extend module 2019-01-02 13:02:06 -07:00
uplink_status_propagation_db.py [SR-IOV] Default "propagate_uplink_status" flag to True 2022-03-21 11:43:08 +00:00
vlantransparent_db.py remove the neutron.db._resource_extend module 2019-01-02 13:02:06 -07:00