openstack
/
neutron
Code Issues Proposed changes

Change SG rules backref load method to "joined" 

Since [1], we introduced a way to skip the load of the OVO synthetic
fields depending on the resource fields retrieved. In the case of the
security groups (SG), the SG rules are child objects to the SGs. The SG
rules are retrieved when a SG OVO is created.

The improvement done in [1] is to make the SG rules load dynamically,
that means using the load mode "lazy='dynamic'". That will issue a SQL
query only if the SG rules are read; if not, the query is never issued.

However since [2] (and this is previous to the [1] optimization), we
always add the field "shared" to the filters and thus to the fields to
retrieve, because it is a policy required field. Because "shared" is a
synthetic field [3], that will force the SG "synthetic_fields" load and
the retrieval of the SG rules always. This is undoing any performance
improvement.

This patch is changing the loading method to "joined"; that will request
the SG rules in the same SQL query, instead of issuing separate queries
for the SG rules. Until a method to load the SG "shared" field,
independently of the synthetic OVO fields is implemented, this change
will improve the SG retrieval performance. In a testing environment
with around 5500K SG and 4 rules (default ones) per SG:
* lazy='dynamic': 38 seconds
* lazy='select': 20 seconds
* lazy='joined': 12 seconds

[1]https://review.opendev.org/q/topic:%22bug/1810563%22
[2]https://review.opendev.org/c/openstack/neutron/+/328313
[3]b85b19e384/neutron/objects/rbac_db.py (L349)

Related-Bug: #2052419
Change-Id: I300464472f2348d148f2a3ddac384c883d9d815b
(cherry picked from commit 52662cad7a)
This commit is contained in:
Rodolfo Alonso Hernandez 2024-02-08 21:56:06 +00:00
parent f26a625f80
commit 3dec91ec58
2 changed files with 9 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
neutron/db/models/securitygroup.py
View File

@ -101,9 +101,13 @@ class SecurityGroupRule(standard_attr.HasStandardAttributes, model_base.BASEV2,
port_range_min = sa.Column(sa.Integer)
port_range_max = sa.Column(sa.Integer)
remote_ip_prefix = sa.Column(sa.String(255))
# NOTE(ralonsoh): loading method is temporarily changed to "joined" until
# a proper way to only load the security groups "shared" field, without
# loading the rest of the synthetic fields, is implemented. LP#2052419
# description for more information and context.
security_group = orm.relationship(
SecurityGroup, load_on_pending=True,
backref=orm.backref('rules', cascade='all,delete', lazy='dynamic'),
backref=orm.backref('rules', cascade='all,delete', lazy='joined'),
primaryjoin="SecurityGroup.id==SecurityGroupRule.security_group_id")
source_group = orm.relationship(
SecurityGroup,

4
neutron/tests/unit/db/test_securitygroups_db.py
View File

@ -347,6 +347,10 @@ class SecurityGroupDbMixinTestCase(testlib_api.SqlTestCase):
# Especially we want to check the revision number here.
sg_dict_got = self.mixin.get_security_group(
self.ctx, sg_dict['id'])
# Order the SG rules to avoid issues in the assertion.
for _sg_dict in (sg_dict, sg_dict_got):
_sg_dict['security_group_rules'] = sorted(
_sg_dict['security_group_rules'], key=lambda d: d['id'])
self.assertEqual(sg_dict, sg_dict_got)
def test_security_group_precommit_create_event_with_revisions(self):