Running with a stricter .pylintrc generates a lot of
C0330 warnings (hanging/continued indentation). Fix
the ones in neutron/db.
Trivialfix
Change-Id: I9311cfe5efc51552008072d84aa238e5d0c9de60
The attempt to list security groups for a project, or any
random string, can create a default SG for it. Only allow if
privileges support it.
Closes-bug: #1988026
Change-Id: Ieef7011f48cd2188d4254ff16d90a6465bbabfe3
Do not retrieve the security group port bindings in the database
transaction that deletes a security group. In the previous context,
if a security group port binding is present on the database,
the method raises a ``SecurityGroupInUse``. It is unneeded to
retrieve them again.
This patch also changes the
``SecurityGroupPortBinding.security_group_id`` foreign key. Now if
the security group is deleted, any security group port binding related
will be too, using the database engine. That will ensure no leftover
remains in the database. Although the check done in
"delete_security_group" before the security group is deleted, there is
a minimal possibility of race condition between the first database
transaction (SG port binding check) and the second one (SG deletion).
Trivial-Fix
Change-Id: I1c9c2dd95b98a7cc77509b0d537d7c7766765275
Removed unneeded database context in
``_create_port_security_group_binding``. This method is called always
from inside an active database transaction, when the port is udpated
or created.
The retry context only makes sense when a method is called outside
an active database transaction. It doesn't retry the command if the
context passed has an active transaction.
Trivial-Fix
Change-Id: I8f84c1bce0d1ce2538789e3511fd8f7b64fcd41a
When listing the security group rules, the DB object can be used
instead the OVO. That will save the conversion time. In a
development environment with 1000 SG rules, the DB query took around
100 ms and the same time to load the OVOs from the DB objects.
The ``NeutronDbObject.get_objects`` now can return the DB objects
without converting them to OVO if "return_db_obj=True".
Trivial-Fix
Change-Id: I245032cf99b1b042226703f5ddb4625e176924c5
Add the shared field to security group API responses and support
using shared as a query filter.
A follow-up patch will remove the temporary api def once it is merged
and released in neutron-lib.
Related-Bug: #1942615
Depends-On: https://review.opendev.org/c/openstack/neutron-lib/+/812617
Change-Id: Ic04be8f0b7097c8aed19365f06089aa7af333eb9
The goal of [1] is to, in case of failing when removing the quota
reservation, continue the operation. Any expired reservation will
be removed automatically in any driver.
If the DB transaction fails, it should affect only to the reservation
trying to be deleted. This is why this patch isolates the
"remove_reservation" method and guarantees it is called outside an
active DB session. That guarantees, in case of failure, no other DB
operation will be affected.
This patch also partially reverts [2] but still checks the security
group rule quota when a new security group is created. Instead of
creating and releasing a quota reservation for the security group
rules created, now only the available quota limit is checked before
creating them. That won't prevent another operation to create security
group rules in parallel, exceeding the available quota. However, this
is not even guaranteed with the current quota driver.
[1]https://review.opendev.org/c/openstack/neutron/+/805031
[2]https://review.opendev.org/c/openstack/neutron/+/701565
Closes-Bug: #1943714
Change-Id: Id73368576a948f78a043d7cf0be16661a65626a9
SECURITY_GROUP and SECURITY_GROUP_RULE resources now
is using payload style callback. Shim can be removed
Change-Id: Icd8e61100b5e2db24a737e852441548eca8f8424
With new scopes, when e.g. project admin context is used to ensure
default SG for different tenant, elevated context needs to be used
to make db query. Otherwise default SG will not be found and attempt to
create it in DB may fail with DuplicateDbEntry error.
Closes-Bug: #1938910
Change-Id: Ib884be6aa12bd0d3faf83f3e753f8e7aad503b68
The quota driver ``ConfDriver`` was deprecated in Liberty release.
``NullQuotaDriver`` is created for testing although it could be used
in production if no quota enforcement is needed. However, because
the Quota engine is not plugable (is an extension always loaded), it
could be interesting to make it plugable as any other plugin.
This patch also creates a Quota engine driver API class that should be
used in any Quota engine driver. Currently it is used in the three
in-tree drivers implemented: ``NullQuotaDriver``, ``DbQuotaDriver``
and ``DbQuotaNoLockDriver``.
Change-Id: Ib4af80e18fac52b9f68f26c84a215415e63c2822
Closes-Bug: #1928211
With new system/project scopes it may happen that project admin
will try to list security groups for different project and during
that call Neutron will ensure that default security group is created.
In such case elevated context needs to be used to get SG object from
the database otherwise, SG will not be found and error 500 will be
returned through the API.
Use of elevated context is fine here as if user don't have access to
the SG, it will be filtered out by policy mechanism and it will not
be returned through API.
Closes-Bug: #1934115
Change-Id: I0ca07d1a1aaf05c1992aea9e29575580d7933324
This patch switches over to callback payloads for SECURITY_GROUP
events. To do so a few shims are put into place the handle both
payload and kwarg style callbacks; these shims will be removed once
all events use payloads. In addition a few UT updates are included to
get the tests working properly with payloads.
Change-Id: I6161a8b387812808c4d679f882a3193c93235647
This patch switches the code over to the payload style of callbacks
for PORT BEFORE_CREATE and PRECOMMIT_CREATE events
Change-Id: Ia6903be1af4de575ab9d82c9cb0c88290d07abb3
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
removed.
NeutronLibImpact
Change-Id: I889364b5d184d47a79fe6ed604ce13a4b334acfa
Getting default SG for a project is quite a frequent operation
that happens for example on each port create/update events.
Since default SG id does not change often for a project/tenant
(if at all) it makes sense to cache it in memory to avoid DB access
each time.
Change-Id: I2493eb65ddd548435cf846e1ae7ac65ce8bc14a4
If security group API is disabled, there is no point to create default
security group for tenant when e.g. network is created.
Closes-Bug: #1913297
Change-Id: Ib73babdd563e3e8c21ce6f63456cc87af414c5aa
New API extension was added in [1] to extend security group rules with
"normalized_cidr" read only attribute.
This patch implements this API extension in Neutron ML2 plugin and
extends security group rules with "normalized_cidr" value.
[1] https://review.opendev.org/#/c/743630/
Related-Bug: #1869129
Change-Id: I65584817a22f952da8da979ab68cd6cfaa2143be
- Add api extension and db model changes to support remote_address_group_id
in SG rules.
- RPC and firewall agent changes will be in the follow-up patches.
Change-Id: I99681736d05eefd82bdba72b3866eab9468ef5dd
Implements: blueprint address-groups-in-sg-rules
This patch implements the last code bits pending to
conclude the new DB engine facade migration.
Due to the resultant interactions in the modified code, is
not possible to submit smaller patches; this code must be
migrated at once.
Partially-Implements blueprint: enginefacade-switch
Signed-off-by: Slawek Kaplonski <skaplons@redhat.com>
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>
Change-Id: Id3f09b78c8d0a8daa7ec4fa6f5bf79f7d5ab8f8b
Included standard attributes ID in some OVO dictionaries to improve
the OVN revision numbers operation. Having this ID, there is no need
to retrieve it from the database.
The following OVOs have been updated:
- port
- network
- subnet
- security group
- security group rule
- router
Closes-Bug: #1904188
Change-Id: Ia16335a2fc8f9324b9489692c76a73e4ef5bef96
This patch adds the verification of whether admin context when
verifying the valid security groups of port.
Change-Id: I2674bdc448d9a091b9fe8c68f0866fd19141c6be
Closes-Bug: #1890539
As spotted in Focal testing patch [0], pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 [1]
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
[0] https://review.opendev.org/#/c/738163/
[1] https://github.com/PyCQA/pylint/issues/2737
Change-Id: Ie646b7093aa8634fd950c136a0eba9adcf56591c
Method _ensure_default_security_group wasn't atomic as it first tries to get
default SG and if that not exists in DB, it tries to create it.
It may happend, like e.g. in Calico plugin that between
get_default_sg_id method and create_security_group method, this default
SG will be created by other neutron worker. And in such case there will
be Duplicate entry exception raised.
So this patch is adding handling of such exception.
Change-Id: I515c310f221e7d9ae3be59a26260538d1bc591c2
Closes-Bug: #1883730
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Change-Id: I5208dc41bff1983ecd323286f427296b722da62a
Allow the subscriber to know the deleted security group name. It can
help some downstream callback utilization to use the deleted sg name for
additional workflow.
Change-Id: Ia321ff96cf445d20f082779d3f6a96fac07b0943
now filter security groups by tenant_id, will auto ensure default security
group, but project_id not. tenant_id is deprecated, we are used to filtering
by project_id.
Closes-bug: #1875849
Change-Id: I3f1a2698768e2d66811bb290a892a1b089dff4ed
During processing of security group rule list API call Neutron will
now ensure that default security group for project given in the filters
or in the context exists.
It is similar to what is done for list of security groups or creation of
new network/port in the project.
Change-Id: Id6fee5a752968b356b884d939b708a420016c9bc
Closes-Bug: #1864171
Retrieving the SG rules now is used the admin context. This allows to
get all possible rules, independently of the user calling. The filters
passed and the RBAC policies filter those results, returning only:
- The SG rules belonging to the user.
- The SG rules belonging to a SG owned by the user.
However, if the SG list is too long, the query can take a lot of time.
Instead of this, the filtering is done in the DB query. If no filters
are passed to "get_security_group_rules" and the context is not the
admin context, only the rules specified in the first paragraph will
be retrieved.
Because overwriting the method "get_objects" is too complex, an
intermediate query is done to retrieve the SG rule IDs. Those IDs
will be used as a filter in the "get_objects" call.
Closes-Bug: #1863201
Change-Id: I25d3da929f8d0b6ee15d7b90ec59b9d58a4ae6a5
This method was defined in SecurityGroupDbMixin class but it seems that
it isn't used anywhere around Neutron and stadium projects.
So we can simply remove it.
Change-Id: I5df04d471af7f895dcf55e3233615b89acacbc6f
The tracked resources quota check is done at the beginning of an API
call to the Neutron server. The API call contains a resource and an
action over the resource. In case of creation, the server checks if
the number of items requested fits in the existing quota.
In case of security group creation, the tracked resource checked is
"security_group". But "SecurityGroupDbMixin.create_security_group"
method also creates several default rules for the new group and the
quota for "security_group_rule" is not enforced.
This patch checks the number of security group rules being created
("delta") and checks in the plugin method (not in the API method) if
there is enough room for those new rules (tracked resource
"security_group_rule").
Change-Id: I0a9b91b09d6260ff96fdba2f0a455de53bbc1f00
Closes-Bug: #1858680
In case when user's security group contains rules created e.g.
by admin, and such rules has got admin's tenant as tenant_id,
owner of security group should be able to see those rules.
Some time ago this was addressed for request:
GET /v2.0/security-groups/<sec_group_id>
But it is also required to behave in same way for
GET /v2.0/security-group-rules
So this patch fixes this behaviour for listing of security
group rules.
To achieve that this patch also adds new policy rule:
ADMIN_OWNER_OR_SG_OWNER which is similar to already existing
ADMIN_OWNER_OR_NETWORK_OWNER used e.g. for listing or creating
ports.
Change-Id: I09114712582d2d38d14cf1683b87a8ce3a8e8c3c
Closes-Bug: #1824248
A security group rule where port_range_min:port_range_max
is 1:65535 is specifying all ports, but it is not optimal
for backends to try and implement this potentially large
rule.
Since it is essentially the entire port range, change
min:max to be None, making the rule specify the entire
protocol instead.
Change-Id: Iff22e2fc84d679e20a5a04b8516750c6ea949078
Closes-bug: #1848213
The security group code was changed recently to always
normalize IPv6 ICMP protocol names to 'ipv6-icmp', but it
did not cover when a number is used instead. Normalize
protocol number 1 to 58 for IPv6 ICMP as well.
Change-Id: Ife8263196f3d678d8455f07834c9f6c1330acc00
Closes-bug: #1582500
Bulk ports were not binding correctly when they were created. This is
due to a few inconsistencies between the create_port code and the
create_port_bulk code, mostly a result of the use of the Port object in
the bulk code.
Change-Id: I3bcd3cec12b1b6f6a568cda4bfeb569f636efb98
Closes-Bug: #1835209
This patch switches over to neutron-lib's version of the the
_AuthenticBase, AuthenticEUI and AuthenticIPNetwork classes by
deleting them from neutron and using lib's version instead.
Depends-On: https://review.opendev.org/#/c/659881/
NeutronLibImpact
Change-Id: Ia3d3db401d6abcb9c9965b945bcd4c199f8e812b
Brian Haley