The attempt to list security groups for a project, or any
random string, can create a default SG for it. Only allow if
privileges support it.
Do not retrieve the security group port bindings in the database
transaction that deletes a security group. In the previous context,
if a security group port binding is present on the database,
the method raises a ``SecurityGroupInUse``. It is unneeded to
retrieve them again.
This patch also changes the
``SecurityGroupPortBinding.security_group_id`` foreign key. Now if
the security group is deleted, any security group port binding related
will be too, using the database engine. That will ensure no leftover
remains in the database. Although the check done in
"delete_security_group" before the security group is deleted, there is
a minimal possibility of race condition between the first database
transaction (SG port binding check) and the second one (SG deletion).
Removed unneeded database context in
``_create_port_security_group_binding``. This method is called always
from inside an active database transaction, when the port is udpated
The retry context only makes sense when a method is called outside
an active database transaction. It doesn't retry the command if the
context passed has an active transaction.
When listing the security group rules, the DB object can be used
instead the OVO. That will save the conversion time. In a
development environment with 1000 SG rules, the DB query took around
100 ms and the same time to load the OVOs from the DB objects.
The ``NeutronDbObject.get_objects`` now can return the DB objects
without converting them to OVO if "return_db_obj=True".
Add the shared field to security group API responses and support
using shared as a query filter.
A follow-up patch will remove the temporary api def once it is merged
and released in neutron-lib.
The goal of  is to, in case of failing when removing the quota
reservation, continue the operation. Any expired reservation will
be removed automatically in any driver.
If the DB transaction fails, it should affect only to the reservation
trying to be deleted. This is why this patch isolates the
"remove_reservation" method and guarantees it is called outside an
active DB session. That guarantees, in case of failure, no other DB
operation will be affected.
This patch also partially reverts  but still checks the security
group rule quota when a new security group is created. Instead of
creating and releasing a quota reservation for the security group
rules created, now only the available quota limit is checked before
creating them. That won't prevent another operation to create security
group rules in parallel, exceeding the available quota. However, this
is not even guaranteed with the current quota driver.
With new scopes, when e.g. project admin context is used to ensure
default SG for different tenant, elevated context needs to be used
to make db query. Otherwise default SG will not be found and attempt to
create it in DB may fail with DuplicateDbEntry error.
The quota driver ``ConfDriver`` was deprecated in Liberty release.
``NullQuotaDriver`` is created for testing although it could be used
in production if no quota enforcement is needed. However, because
the Quota engine is not plugable (is an extension always loaded), it
could be interesting to make it plugable as any other plugin.
This patch also creates a Quota engine driver API class that should be
used in any Quota engine driver. Currently it is used in the three
in-tree drivers implemented: ``NullQuotaDriver``, ``DbQuotaDriver``
With new system/project scopes it may happen that project admin
will try to list security groups for different project and during
that call Neutron will ensure that default security group is created.
In such case elevated context needs to be used to get SG object from
the database otherwise, SG will not be found and error 500 will be
returned through the API.
Use of elevated context is fine here as if user don't have access to
the SG, it will be filtered out by policy mechanism and it will not
be returned through API.
This patch switches over to callback payloads for SECURITY_GROUP
events. To do so a few shims are put into place the handle both
payload and kwarg style callbacks; these shims will be removed once
all events use payloads. In addition a few UT updates are included to
get the tests working properly with payloads.
This patch switches over to the payload style of callbacks for
NETWORK based events. As part of this change a few shims are needed
to handle cases where some callbacks don't yet use payloads and others
do. Once we move over to payloads for all callbacks the shims can be
Getting default SG for a project is quite a frequent operation
that happens for example on each port create/update events.
Since default SG id does not change often for a project/tenant
(if at all) it makes sense to cache it in memory to avoid DB access
New API extension was added in  to extend security group rules with
"normalized_cidr" read only attribute.
This patch implements this API extension in Neutron ML2 plugin and
extends security group rules with "normalized_cidr" value.
- Add api extension and db model changes to support remote_address_group_id
in SG rules.
- RPC and firewall agent changes will be in the follow-up patches.
Implements: blueprint address-groups-in-sg-rules
This patch implements the last code bits pending to
conclude the new DB engine facade migration.
Due to the resultant interactions in the modified code, is
not possible to submit smaller patches; this code must be
migrated at once.
Partially-Implements blueprint: enginefacade-switch
Signed-off-by: Slawek Kaplonski <email@example.com>
Co-Authored-By: Rodolfo Alonso Hernandez <firstname.lastname@example.org>
Included standard attributes ID in some OVO dictionaries to improve
the OVN revision numbers operation. Having this ID, there is no need
to retrieve it from the database.
The following OVOs have been updated:
- security group
- security group rule
As spotted in Focal testing patch , pep8 test fails with many
C0321 false-positives, reported in pylint as current version does not
support python 3.8 
Use a newer version of pylint and astroid, fixing or disabling some of
the new checks: no-else-*, unnecessary-comprehension, import-outside-toplevel
Method _ensure_default_security_group wasn't atomic as it first tries to get
default SG and if that not exists in DB, it tries to create it.
It may happend, like e.g. in Calico plugin that between
get_default_sg_id method and create_security_group method, this default
SG will be created by other neutron worker. And in such case there will
be Duplicate entry exception raised.
So this patch is adding handling of such exception.
With python 3.x, six.text_type and six.string_type
are just str.
Also removed a six.integer_type since it was the only
one left in a file.
Another step in removing all of six usage from neutron.
Allow the subscriber to know the deleted security group name. It can
help some downstream callback utilization to use the deleted sg name for
now filter security groups by tenant_id, will auto ensure default security
group, but project_id not. tenant_id is deprecated, we are used to filtering
During processing of security group rule list API call Neutron will
now ensure that default security group for project given in the filters
or in the context exists.
It is similar to what is done for list of security groups or creation of
new network/port in the project.
Retrieving the SG rules now is used the admin context. This allows to
get all possible rules, independently of the user calling. The filters
passed and the RBAC policies filter those results, returning only:
- The SG rules belonging to the user.
- The SG rules belonging to a SG owned by the user.
However, if the SG list is too long, the query can take a lot of time.
Instead of this, the filtering is done in the DB query. If no filters
are passed to "get_security_group_rules" and the context is not the
admin context, only the rules specified in the first paragraph will
Because overwriting the method "get_objects" is too complex, an
intermediate query is done to retrieve the SG rule IDs. Those IDs
will be used as a filter in the "get_objects" call.
This method was defined in SecurityGroupDbMixin class but it seems that
it isn't used anywhere around Neutron and stadium projects.
So we can simply remove it.
The tracked resources quota check is done at the beginning of an API
call to the Neutron server. The API call contains a resource and an
action over the resource. In case of creation, the server checks if
the number of items requested fits in the existing quota.
In case of security group creation, the tracked resource checked is
"security_group". But "SecurityGroupDbMixin.create_security_group"
method also creates several default rules for the new group and the
quota for "security_group_rule" is not enforced.
This patch checks the number of security group rules being created
("delta") and checks in the plugin method (not in the API method) if
there is enough room for those new rules (tracked resource
In case when user's security group contains rules created e.g.
by admin, and such rules has got admin's tenant as tenant_id,
owner of security group should be able to see those rules.
Some time ago this was addressed for request:
But it is also required to behave in same way for
So this patch fixes this behaviour for listing of security
To achieve that this patch also adds new policy rule:
ADMIN_OWNER_OR_SG_OWNER which is similar to already existing
ADMIN_OWNER_OR_NETWORK_OWNER used e.g. for listing or creating
A security group rule where port_range_min:port_range_max
is 1:65535 is specifying all ports, but it is not optimal
for backends to try and implement this potentially large
Since it is essentially the entire port range, change
min:max to be None, making the rule specify the entire
The security group code was changed recently to always
normalize IPv6 ICMP protocol names to 'ipv6-icmp', but it
did not cover when a number is used instead. Normalize
protocol number 1 to 58 for IPv6 ICMP as well.
Bulk ports were not binding correctly when they were created. This is
due to a few inconsistencies between the create_port code and the
create_port_bulk code, mostly a result of the use of the Port object in
the bulk code.
This patch switches over to neutron-lib's version of the the
_AuthenticBase, AuthenticEUI and AuthenticIPNetwork classes by
deleting them from neutron and using lib's version instead.