New Quota driver `DbQuotaNoLockDriver`
This new quota driver, ``DbQuotaNoLockDriver``, does not create a lock per (resource, project_id) but retrieves the instant (resource, project_id) usage and the current (resource, project_id) reservations. If the requested number of resources fit the available quota, a new ``Reservation`` register is created with the amount of units requested. All those operations are done inside a DB transaction context. That means the amount of resources and reservations is guaranteed inside this transaction (depending on the DB backend isolation level defined) and the new reservation created will not clash with other DB transation. That will guarantee the number of resources and instant reservations never exceed the quota limits defined for this (resource, project_id). NOTES: - This change tries to be as unobtrusive as possible. The new driver uses the same ``DbQuotaDriver`` dabatase tables (except for ``QuotaUsage``) and the same Quota engine API, located in ``neutron.quota``. However, the Quota engine resources implements some particular API actions like "dirty", that are not used in the new driver. - The Pecan Quota enforcement hooks, ``neutron.pecan_wgsi.hooks.quota_enforcement``, execute actions like "resync", "mark_resources_dirty" or "set_resources_dirty", that has no meaning in the new driver. - The isolation between the Quota engine and the Pecan hook, and the driver itself is not clearly defined. A refactor of the Quota engine, Quota service, Quota drivers and a common API between the driver and the engine is needed. - If ``DbQuotaDriver`` is deprecated, ``CountableResource`` and ``TrackedResource`` will be joined in a single class. This resource class will have a count method (countable) or a hard dependency on a database table (tracked resource). The only difference will be the "count" method implementation. Closes-Bug: #1926787 Change-Id: I4f98c6fcd781459fd7150aff426d19c7fdfa98c1
This commit is contained in:
Rodolfo Alonso Hernandez
Rodolfo Alonso
parent
9e6b7a2284
commit
e135a8221d
@ -43,7 +43,7 @@ limits are currently not enforced on RPC interfaces listening on the AMQP
|
||||
bus.
|
||||
|
||||
Plugin and ML2 drivers are not supposed to enforce quotas for resources they
|
||||
manage. However, the subnet_allocation [#]_ extension is an exception and will
|
||||
manage. However, the subnet_allocation [1]_ extension is an exception and will
|
||||
be discussed below.
|
||||
|
||||
The quota management and enforcement mechanisms discussed here apply to every
|
||||
@ -59,12 +59,14 @@ There are two main components in the Neutron quota system:
|
||||
* The Quota Engine.
|
||||
|
||||
Both components rely on a quota driver. The neutron codebase currently defines
|
||||
two quota drivers:
|
||||
three quota drivers:
|
||||
|
||||
* neutron.db.quota.driver.DbQuotaDriver
|
||||
* neutron.db.quota.driver_nolock.DbQuotaNoLockDriver (default)
|
||||
* neutron.quota.ConfDriver
|
||||
|
||||
The latter driver is however deprecated.
|
||||
The latter driver is however deprecated. The ``DbQuotaNoLockDriver`` is the
|
||||
default quota driver, defined in the configuration option ``quota_driver``.
|
||||
|
||||
The Quota API extension handles quota management, whereas the Quota Engine
|
||||
component handles quota enforcement. This API extension is loaded like any
|
||||
@ -91,7 +93,7 @@ For a reservation to be successful, the total amount of resources requested,
|
||||
plus the total amount of resources reserved, plus the total amount of resources
|
||||
already stored in the database should not exceed the project's quota limit.
|
||||
|
||||
Finally, both quota management and enforcement rely on a "quota driver" [#]_,
|
||||
Finally, both quota management and enforcement rely on a "quota driver" [2]_,
|
||||
whose task is basically to perform database operations.
|
||||
|
||||
Quota Management
|
||||
@ -100,14 +102,14 @@ Quota Management
|
||||
The quota management component is fairly straightforward.
|
||||
|
||||
However, unlike the vast majority of Neutron extensions, it uses it own
|
||||
controller class [#]_.
|
||||
controller class [3]_.
|
||||
This class does not implement the POST operation. List, get, update, and
|
||||
delete operations are implemented by the usual index, show, update and
|
||||
delete methods. These method simply call into the quota driver for either
|
||||
fetching project quotas or updating them.
|
||||
|
||||
The _update_attributes method is called only once in the controller lifetime.
|
||||
This method dynamically updates Neutron's resource attribute map [#]_ so that
|
||||
This method dynamically updates Neutron's resource attribute map [4]_ so that
|
||||
an attribute is added for every resource managed by the quota engine.
|
||||
Request authorisation is performed in this controller, and only 'admin' users
|
||||
are allowed to modify quotas for projects. As the neutron policy engine is not
|
||||
@ -131,11 +133,18 @@ Resource Usage Info
|
||||
Neutron has two ways of tracking resource usage info:
|
||||
|
||||
* CountableResource, where resource usage is calculated every time quotas
|
||||
limits are enforced by counting rows in the resource table and reservations
|
||||
for that resource.
|
||||
* TrackedResource, which instead relies on a specific table tracking usage
|
||||
data, and performs explicitly counting only when the data in this table are
|
||||
not in sync with actual used and reserved resources.
|
||||
limits are enforced by counting rows in the resource table or resources
|
||||
tables and reservations for that resource.
|
||||
* TrackedResource, depends on the selected driver:
|
||||
|
||||
* DbQuotaDriver: the resource usage relies on a specific table tracking
|
||||
usage data, and performs explicitly counting only when the data in this
|
||||
table are not in sync with actual used and reserved resources.
|
||||
* DbQuotaNoLockDriver: the resource usage is counted directly from the
|
||||
database table associated to the resource. In this new driver,
|
||||
CountableResource and TrackedResource could look similar but
|
||||
TrackedResource depends on one single database model (table) and the
|
||||
resource count is done directly on this table only.
|
||||
|
||||
Another difference between CountableResource and TrackedResource is that the
|
||||
former invokes a plugin method to count resources. CountableResource should be
|
||||
@ -147,6 +156,9 @@ use CountableResource instances.
|
||||
Resource creation is performed by the create_resource_instance factory method
|
||||
in the neutron.quota.resource module.
|
||||
|
||||
DbQuotaDriver description
|
||||
-------------------------
|
||||
|
||||
From a performance perspective, having a table tracking resource usage
|
||||
has some advantages, albeit not fundamental. Indeed the time required for
|
||||
executing queries to explicitly count objects will increase with the number of
|
||||
@ -183,15 +195,48 @@ caveats' section, it is more reliable than solutions such as:
|
||||
* Having a periodic task synchronising quota usage data with actual data in
|
||||
the Neutron DB.
|
||||
|
||||
Finally, regardless of whether CountableResource or TrackedResource is used,
|
||||
the quota engine always invokes its count() method to retrieve resource usage.
|
||||
|
||||
DbQuotaNoLockDriver description
|
||||
-------------------------------
|
||||
|
||||
The strategy of this quota driver is the opposite to ``DbQuotaDriver``.
|
||||
Instead of tracking the usage quota of each resource in a specific table,
|
||||
this driver retrieves the used resources directly form the database.
|
||||
Each TrackedResource is linked to a database table that stores the tracked
|
||||
resources. This driver claims that a trivial query on the resource table,
|
||||
filtering by project ID, is faster than attending to the DB events and tracking
|
||||
the quota usage in an independent table.
|
||||
|
||||
This driver relays on the database engine transactionality isolation. Each
|
||||
time a new resource is requested, the quota driver opens a database transaction
|
||||
to:
|
||||
|
||||
* Clean up the expired reservations. The amount of expired reservations is
|
||||
always limited because of the short timeout set (2 minutes).
|
||||
* Retrieve the used resources for a specific project. This query retrieves
|
||||
only the "project_id" column of the resource to avoid backref requests; that
|
||||
limits the scope of the query and speeds up it.
|
||||
* Retrieve the reserved resources, created by other concurrent operations.
|
||||
* If there is enough quota, create a new reservation register.
|
||||
|
||||
Those operations, executed in the same transaction, are fast enough to avoid
|
||||
another concurrent resource reservation, exceeding the available quota. At the
|
||||
same time, this driver does not create a lock per resource and project ID,
|
||||
allowing concurrent requests that won't be blocked by the resource lock.
|
||||
Because the quota reservation process, described before, is a fast operation,
|
||||
the chances of overcommiting resources over the quota limits are low. Neutron
|
||||
does not enforce quota in such way that a quota limit violation could never
|
||||
occur [5]_.
|
||||
|
||||
Regardless of whether CountableResource or TrackedResource is used, the quota
|
||||
engine always invokes its count() method to retrieve resource usage.
|
||||
Therefore, from the perspective of the Quota engine there is absolutely no
|
||||
difference between CountableResource and TrackedResource.
|
||||
|
||||
Quota Enforcement
|
||||
-----------------
|
||||
Quota Enforcement in DbQuotaDriver
|
||||
----------------------------------
|
||||
|
||||
Before dispatching a request to the plugin, the Neutron 'base' controller [#]_
|
||||
Before dispatching a request to the plugin, the Neutron 'base' controller [6]_
|
||||
attempts to make a reservation for requested resource(s).
|
||||
Reservations are made by calling the make_reservation method in
|
||||
neutron.quota.QuotaEngine.
|
||||
@ -228,7 +273,7 @@ While non-locking approaches are possible, it has been found out that, since
|
||||
a non-locking algorithms increases the chances of collision, the cost of
|
||||
handling a DBDeadlock is still lower than the cost of retrying the operation
|
||||
when a collision is detected. A study in this direction was conducted for
|
||||
IP allocation operations, but the same principles apply here as well [#]_.
|
||||
IP allocation operations, but the same principles apply here as well [7]_.
|
||||
Nevertheless, moving away for DB-level locks is something that must happen
|
||||
for quota enforcement in the future.
|
||||
|
||||
@ -345,9 +390,10 @@ Please be aware of the following limitations of the quota enforcement engine:
|
||||
References
|
||||
----------
|
||||
|
||||
.. [#] Subnet allocation extension: http://opendev.org/openstack/neutron/tree/neutron/extensions/subnetallocation.py
|
||||
.. [#] DB Quota driver class: http://opendev.org/openstack/neutron/tree/neutron/db/quota/driver.py#n30
|
||||
.. [#] Quota API extension controller: http://opendev.org/openstack/neutron/tree/neutron/extensions/quotasv2.py#n40
|
||||
.. [#] Neutron resource attribute map: http://opendev.org/openstack/neutron/tree/neutron/api/v2/attributes.py#n639
|
||||
.. [#] Base controller class: http://opendev.org/openstack/neutron/tree/neutron/api/v2/base.py#n50
|
||||
.. [#] http://lists.openstack.org/pipermail/openstack-dev/2015-February/057534.html
|
||||
.. [1] Subnet allocation extension: http://opendev.org/openstack/neutron/tree/neutron/extensions/subnetallocation.py
|
||||
.. [2] DB Quota driver class: http://opendev.org/openstack/neutron/tree/neutron/db/quota/driver.py#n30
|
||||
.. [3] Quota API extension controller: http://opendev.org/openstack/neutron/tree/neutron/extensions/quotasv2.py#n40
|
||||
.. [4] Neutron resource attribute map: http://opendev.org/openstack/neutron/tree/neutron/api/v2/attributes.py#n639
|
||||
.. [5] Quota limit exceeded: https://bugs.launchpad.net/neutron/+bug/1862050/
|
||||
.. [6] Base controller class: http://opendev.org/openstack/neutron/tree/neutron/api/v2/base.py#n50
|
||||
.. [7] http://lists.openstack.org/pipermail/openstack-dev/2015-February/057534.html
|
||||
|
||||
@ -18,8 +18,9 @@ from oslo_config import cfg
|
||||
|
||||
from neutron._i18n import _
|
||||
|
||||
QUOTA_DB_MODULE = 'neutron.db.quota.driver'
|
||||
QUOTA_DB_DRIVER = '%s.DbQuotaDriver' % QUOTA_DB_MODULE
|
||||
|
||||
QUOTA_DB_MODULE = 'neutron.db.quota.driver_nolock'
|
||||
QUOTA_DB_DRIVER = QUOTA_DB_MODULE + '.DbQuotaNoLockDriver'
|
||||
QUOTA_CONF_DRIVER = 'neutron.quota.ConfDriver'
|
||||
QUOTAS_CFG_GROUP = 'QUOTAS'
|
||||
|
||||
|
||||
78
neutron/db/quota/driver_nolock.py
Normal file
78
neutron/db/quota/driver_nolock.py
Normal file
@ -0,0 +1,78 @@
|
||||
# Copyright (c) 2021 Red Hat Inc.
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from neutron_lib.db import api as db_api
|
||||
from neutron_lib import exceptions
|
||||
from oslo_log import log
|
||||
|
||||
from neutron.db.quota import api as quota_api
|
||||
from neutron.db.quota import driver as quota_driver
|
||||
|
||||
|
||||
LOG = log.getLogger(__name__)
|
||||
|
||||
|
||||
class DbQuotaNoLockDriver(quota_driver.DbQuotaDriver):
|
||||
"""Driver to enforce quotas and retrieve quota information
|
||||
|
||||
This driver does not use a (resource, project_id) lock but relays on the
|
||||
simplicity of the calculation method, that is executed in a single database
|
||||
transaction. The method (1) counts the number of created resources and (2)
|
||||
the number of resource reservations. If the requested number of resources
|
||||
do not exceeds the quota, a new reservation register is created.
|
||||
|
||||
This calculation method does not guarantee the quota enforcement if, for
|
||||
example, the database isolation level is read committed; two transactions
|
||||
can count the same number of resources and reservations, committing both
|
||||
a new reservation exceeding the quota. But the goal of this reservation
|
||||
method is to be fast enough to avoid the concurrency when counting the
|
||||
resources while not blocking concurrent API operations.
|
||||
"""
|
||||
@db_api.retry_if_session_inactive()
|
||||
def make_reservation(self, context, project_id, resources, deltas, plugin):
|
||||
resources_over_limit = []
|
||||
with db_api.CONTEXT_WRITER.using(context):
|
||||
# Filter out unlimited resources.
|
||||
limits = self.get_tenant_quotas(context, resources, project_id)
|
||||
unlimited_resources = set([resource for (resource, limit) in
|
||||
limits.items() if limit < 0])
|
||||
requested_resources = (set(deltas.keys()) - unlimited_resources)
|
||||
|
||||
# Delete expired reservations before counting valid ones. This
|
||||
# operation is fast and by calling it before making any
|
||||
# reservation, we ensure the freshness of the reservations.
|
||||
quota_api.remove_expired_reservations(context,
|
||||
tenant_id=project_id)
|
||||
|
||||
# Count the number of (1) used and (2) reserved resources for this
|
||||
# project_id. If any resource limit is exceeded, raise exception.
|
||||
for resource_name in requested_resources:
|
||||
tracked_resource = resources.get(resource_name)
|
||||
if not tracked_resource:
|
||||
continue
|
||||
|
||||
used_and_reserved = tracked_resource.count(
|
||||
context, None, project_id, count_db_registers=True)
|
||||
resource_num = deltas[resource_name]
|
||||
if limits[resource_name] < (used_and_reserved + resource_num):
|
||||
resources_over_limit.append(resource_name)
|
||||
|
||||
if resources_over_limit:
|
||||
raise exceptions.OverQuota(overs=sorted(resources_over_limit))
|
||||
|
||||
return quota_api.create_reservation(context, project_id, deltas)
|
||||
|
||||
def cancel_reservation(self, context, reservation_id):
|
||||
quota_api.remove_reservation(context, reservation_id, set_dirty=False)
|
||||
@ -38,7 +38,7 @@ DEFAULT_QUOTAS_ACTION = 'default'
|
||||
RESOURCE_NAME = 'quota'
|
||||
RESOURCE_COLLECTION = RESOURCE_NAME + "s"
|
||||
QUOTAS = quota.QUOTAS
|
||||
DB_QUOTA_DRIVER = 'neutron.db.quota.driver.DbQuotaDriver'
|
||||
DB_QUOTA_DRIVER = cfg.CONF.QUOTAS.quota_driver
|
||||
EXTENDED_ATTRIBUTES_2_0 = {
|
||||
RESOURCE_COLLECTION: {}
|
||||
}
|
||||
|
||||
@ -23,6 +23,8 @@ from oslo_config import cfg
|
||||
from neutron._i18n import _
|
||||
from neutron.api import extensions
|
||||
from neutron.api.v2 import resource
|
||||
from neutron.db.quota import driver
|
||||
from neutron.db.quota import driver_nolock
|
||||
from neutron.extensions import quotasv2
|
||||
from neutron.quota import resource_registry
|
||||
|
||||
@ -32,7 +34,11 @@ RESOURCE_NAME = 'quota'
|
||||
ALIAS = RESOURCE_NAME + '_' + DETAIL_QUOTAS_ACTION
|
||||
QUOTA_DRIVER = cfg.CONF.QUOTAS.quota_driver
|
||||
RESOURCE_COLLECTION = RESOURCE_NAME + "s"
|
||||
DB_QUOTA_DRIVER = 'neutron.db.quota.driver.DbQuotaDriver'
|
||||
DB_QUOTA_DRIVERS = tuple('.'.join([klass.__module__, klass.__name__])
|
||||
for klass in (driver.DbQuotaDriver,
|
||||
driver_nolock.DbQuotaNoLockDriver,
|
||||
)
|
||||
)
|
||||
EXTENDED_ATTRIBUTES_2_0 = {
|
||||
RESOURCE_COLLECTION: {}
|
||||
}
|
||||
@ -61,8 +67,7 @@ class Quotasv2_detail(api_extensions.ExtensionDescriptor):
|
||||
|
||||
# Ensure new extension is not loaded with old conf driver.
|
||||
extensions.register_custom_supported_check(
|
||||
ALIAS, lambda: QUOTA_DRIVER == DB_QUOTA_DRIVER,
|
||||
plugin_agnostic=True)
|
||||
ALIAS, lambda: QUOTA_DRIVER in DB_QUOTA_DRIVERS, plugin_agnostic=True)
|
||||
|
||||
@classmethod
|
||||
def get_name(cls):
|
||||
|
||||
@ -12,6 +12,8 @@
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import abc
|
||||
|
||||
from neutron_lib.db import api as db_api
|
||||
from neutron_lib.plugins import constants
|
||||
from neutron_lib.plugins import directory
|
||||
@ -22,6 +24,8 @@ from sqlalchemy import exc as sql_exc
|
||||
from sqlalchemy.orm import session as se
|
||||
|
||||
from neutron._i18n import _
|
||||
from neutron.common import utils as n_utils
|
||||
from neutron.conf import quota as quota_conf
|
||||
from neutron.db.quota import api as quota_api
|
||||
|
||||
LOG = log.getLogger(__name__)
|
||||
@ -55,7 +59,7 @@ def _count_resource(context, collection_name, tenant_id):
|
||||
_('No plugins that support counting %s found.') % collection_name)
|
||||
|
||||
|
||||
class BaseResource(object):
|
||||
class BaseResource(object, metaclass=abc.ABCMeta):
|
||||
"""Describe a single resource for quota checking."""
|
||||
|
||||
def __init__(self, name, flag, plural_name=None):
|
||||
@ -98,6 +102,7 @@ class BaseResource(object):
|
||||
return max(value, -1)
|
||||
|
||||
@property
|
||||
@abc.abstractmethod
|
||||
def dirty(self):
|
||||
"""Return the current state of the Resource instance.
|
||||
|
||||
@ -106,6 +111,10 @@ class BaseResource(object):
|
||||
does not track usage.
|
||||
"""
|
||||
|
||||
@abc.abstractmethod
|
||||
def count(self, context, plugin, project_id, **kwargs):
|
||||
"""Return the total count of this resource"""
|
||||
|
||||
|
||||
class CountableResource(BaseResource):
|
||||
"""Describe a resource where the counts are determined by a function."""
|
||||
@ -143,9 +152,13 @@ class CountableResource(BaseResource):
|
||||
name, flag=flag, plural_name=plural_name)
|
||||
self._count_func = count
|
||||
|
||||
def count(self, context, plugin, tenant_id, **kwargs):
|
||||
@property
|
||||
def dirty(self):
|
||||
return
|
||||
|
||||
def count(self, context, plugin, project_id, **kwargs):
|
||||
# NOTE(ihrachys) _count_resource doesn't receive plugin
|
||||
return self._count_func(context, self.plural_name, tenant_id)
|
||||
return self._count_func(context, self.plural_name, project_id)
|
||||
|
||||
|
||||
class TrackedResource(BaseResource):
|
||||
@ -184,13 +197,21 @@ class TrackedResource(BaseResource):
|
||||
self._model_class = model_class
|
||||
self._dirty_tenants = set()
|
||||
self._out_of_sync_tenants = set()
|
||||
# NOTE(ralonsoh): "DbQuotaNoLockDriver" driver does not need to track
|
||||
# the DB events or resync the resource quota usage.
|
||||
if cfg.CONF.QUOTAS.quota_driver == quota_conf.QUOTA_DB_DRIVER:
|
||||
self._track_resource_events = False
|
||||
else:
|
||||
self._track_resource_events = True
|
||||
|
||||
@property
|
||||
def dirty(self):
|
||||
if not self._track_resource_events:
|
||||
return
|
||||
return self._dirty_tenants
|
||||
|
||||
def mark_dirty(self, context):
|
||||
if not self._dirty_tenants:
|
||||
if not self._dirty_tenants or not self._track_resource_events:
|
||||
return
|
||||
with db_api.CONTEXT_WRITER.using(context):
|
||||
# It is not necessary to protect this operation with a lock.
|
||||
@ -236,7 +257,8 @@ class TrackedResource(BaseResource):
|
||||
return usage_info
|
||||
|
||||
def resync(self, context, tenant_id):
|
||||
if tenant_id not in self._out_of_sync_tenants:
|
||||
if (tenant_id not in self._out_of_sync_tenants or
|
||||
not self._track_resource_events):
|
||||
return
|
||||
LOG.debug(("Synchronizing usage tracker for tenant:%(tenant_id)s on "
|
||||
"resource:%(resource)s"),
|
||||
@ -302,15 +324,37 @@ class TrackedResource(BaseResource):
|
||||
reserved = reservations.get(self.name, 0)
|
||||
return reserved
|
||||
|
||||
def count(self, context, _plugin, tenant_id, resync_usage=True):
|
||||
def count(self, context, _plugin, tenant_id, resync_usage=True,
|
||||
count_db_registers=False):
|
||||
"""Return the count of the resource.
|
||||
|
||||
The _plugin parameter is unused but kept for
|
||||
compatibility with the signature of the count method for
|
||||
CountableResource instances.
|
||||
"""
|
||||
return (self.count_used(context, tenant_id, resync_usage) +
|
||||
self.count_reserved(context, tenant_id))
|
||||
if count_db_registers:
|
||||
count = self._count_db_registers(context, tenant_id)
|
||||
else:
|
||||
count = self.count_used(context, tenant_id, resync_usage)
|
||||
|
||||
return count + self.count_reserved(context, tenant_id)
|
||||
|
||||
def _count_db_registers(self, context, project_id):
|
||||
"""Return the existing resources (self._model_class) in a project.
|
||||
|
||||
The query executed must be as fast as possible. To avoid retrieving all
|
||||
model backref relationship columns, only "project_id" is requested
|
||||
(this column always exists in the DB model because is used in the
|
||||
filter).
|
||||
"""
|
||||
# TODO(ralonsoh): declare the OVO class instead the DB model and use
|
||||
# ``NeutronDbObject.count`` with the needed filters and fields to
|
||||
# retrieve ("project_id").
|
||||
admin_context = n_utils.get_elevated_context(context)
|
||||
with db_api.CONTEXT_READER.using(admin_context):
|
||||
query = admin_context.session.query(self._model_class.project_id)
|
||||
query = query.filter(self._model_class.project_id == project_id)
|
||||
return query.count()
|
||||
|
||||
def _except_bulk_delete(self, delete_context):
|
||||
if delete_context.mapper.class_ == self._model_class:
|
||||
@ -321,12 +365,16 @@ class TrackedResource(BaseResource):
|
||||
self._model_class)
|
||||
|
||||
def register_events(self):
|
||||
if not self._track_resource_events:
|
||||
return
|
||||
listen = db_api.sqla_listen
|
||||
listen(self._model_class, 'after_insert', self._db_event_handler)
|
||||
listen(self._model_class, 'after_delete', self._db_event_handler)
|
||||
listen(se.Session, 'after_bulk_delete', self._except_bulk_delete)
|
||||
|
||||
def unregister_events(self):
|
||||
if not self._track_resource_events:
|
||||
return
|
||||
try:
|
||||
db_api.sqla_remove(self._model_class, 'after_insert',
|
||||
self._db_event_handler)
|
||||
|
||||
@ -51,6 +51,7 @@ from neutron.common import ipv6_utils
|
||||
from neutron.common import test_lib
|
||||
from neutron.common import utils
|
||||
from neutron.conf import policies
|
||||
from neutron.conf import quota as quota_conf
|
||||
from neutron.db import db_base_plugin_common
|
||||
from neutron.db import ipam_backend_mixin
|
||||
from neutron.db.models import l3 as l3_models
|
||||
@ -110,7 +111,10 @@ class NeutronDbPluginV2TestCase(testlib_api.WebTestCase):
|
||||
|
||||
def setUp(self, plugin=None, service_plugins=None,
|
||||
ext_mgr=None):
|
||||
|
||||
quota_conf.register_quota_opts(quota_conf.core_quota_opts, cfg.CONF)
|
||||
cfg.CONF.set_override(
|
||||
'quota_driver', 'neutron.db.quota.driver.DbQuotaDriver',
|
||||
group=quota_conf.QUOTAS_CFG_GROUP)
|
||||
super(NeutronDbPluginV2TestCase, self).setUp()
|
||||
cfg.CONF.set_override('notify_nova_on_port_status_changes', False)
|
||||
cfg.CONF.set_override('allow_overlapping_ips', True)
|
||||
|
||||
@ -30,6 +30,7 @@ from neutron.api.v2 import router
|
||||
from neutron.common import config
|
||||
from neutron.conf import quota as qconf
|
||||
from neutron.db.quota import driver
|
||||
from neutron.db.quota import driver_nolock
|
||||
from neutron import quota
|
||||
from neutron.quota import resource_registry
|
||||
from neutron.tests import base
|
||||
@ -504,29 +505,24 @@ class TestDbQuotaDriver(base.BaseTestCase):
|
||||
|
||||
|
||||
class TestQuotaDriverLoad(base.BaseTestCase):
|
||||
def setUp(self):
|
||||
super(TestQuotaDriverLoad, self).setUp()
|
||||
# Make sure QuotaEngine is reinitialized in each test.
|
||||
quota.QUOTAS._driver = None
|
||||
|
||||
def _test_quota_driver(self, cfg_driver, loaded_driver,
|
||||
with_quota_db_module=True):
|
||||
quota.QUOTAS._driver = None
|
||||
cfg.CONF.set_override('quota_driver', cfg_driver, group='QUOTAS')
|
||||
with mock.patch.dict(sys.modules, {}):
|
||||
if (not with_quota_db_module and
|
||||
'neutron.db.quota.driver' in sys.modules):
|
||||
del sys.modules['neutron.db.quota.driver']
|
||||
quota.QUOTA_DB_MODULE in sys.modules):
|
||||
del sys.modules[quota.QUOTA_DB_MODULE]
|
||||
driver = quota.QUOTAS.get_driver()
|
||||
self.assertEqual(loaded_driver, driver.__class__.__name__)
|
||||
|
||||
def test_quota_db_driver_with_quotas_table(self):
|
||||
self._test_quota_driver('neutron.db.quota.driver.DbQuotaDriver',
|
||||
'DbQuotaDriver', True)
|
||||
def test_quota_driver_load(self):
|
||||
for klass in (quota.ConfDriver, driver.DbQuotaDriver,
|
||||
driver_nolock.DbQuotaNoLockDriver):
|
||||
self._test_quota_driver(
|
||||
'.'.join([klass.__module__, klass.__name__]),
|
||||
klass.__name__, True)
|
||||
|
||||
def test_quota_db_driver_fallback_conf_driver(self):
|
||||
self._test_quota_driver('neutron.db.quota.driver.DbQuotaDriver',
|
||||
'ConfDriver', False)
|
||||
|
||||
def test_quota_conf_driver(self):
|
||||
self._test_quota_driver('neutron.quota.ConfDriver',
|
||||
'ConfDriver', True)
|
||||
def test_quota_driver_fallback_conf_driver(self):
|
||||
self._test_quota_driver(quota.QUOTA_DB_DRIVER, 'ConfDriver', False)
|
||||
|
||||
@ -30,34 +30,44 @@ from neutron.tests.unit import testlib_api
|
||||
|
||||
|
||||
DB_PLUGIN_KLASS = 'neutron.db.db_base_plugin_v2.NeutronDbPluginV2'
|
||||
QUOTA_DRIVER = 'neutron.db.quota.DbQuotaDriver'
|
||||
|
||||
|
||||
meh_quota_flag = 'quota_meh'
|
||||
meh_quota_opts = [cfg.IntOpt(meh_quota_flag, default=99)]
|
||||
|
||||
|
||||
class _BaseResource(resource.BaseResource):
|
||||
|
||||
@property
|
||||
def dirty(self):
|
||||
return False
|
||||
|
||||
def count(self, context, plugin, project_id, **kwargs):
|
||||
pass
|
||||
|
||||
|
||||
class TestResource(base.DietTestCase):
|
||||
"""Unit tests for neutron.quota.resource.BaseResource"""
|
||||
|
||||
def test_create_resource_without_plural_name(self):
|
||||
res = resource.BaseResource('foo', None)
|
||||
res = _BaseResource('foo', None)
|
||||
self.assertEqual('foos', res.plural_name)
|
||||
res = resource.BaseResource('foy', None)
|
||||
res = _BaseResource('foy', None)
|
||||
self.assertEqual('foies', res.plural_name)
|
||||
|
||||
def test_create_resource_with_plural_name(self):
|
||||
res = resource.BaseResource('foo', None,
|
||||
plural_name='foopsies')
|
||||
res = _BaseResource('foo', None, plural_name='foopsies')
|
||||
self.assertEqual('foopsies', res.plural_name)
|
||||
|
||||
def test_resource_default_value(self):
|
||||
res = resource.BaseResource('foo', 'foo_quota')
|
||||
res = _BaseResource('foo', 'foo_quota')
|
||||
with mock.patch('oslo_config.cfg.CONF') as mock_cfg:
|
||||
mock_cfg.QUOTAS.foo_quota = 99
|
||||
self.assertEqual(99, res.default)
|
||||
|
||||
def test_resource_negative_default_value(self):
|
||||
res = resource.BaseResource('foo', 'foo_quota')
|
||||
res = _BaseResource('foo', 'foo_quota')
|
||||
with mock.patch('oslo_config.cfg.CONF') as mock_cfg:
|
||||
mock_cfg.QUOTAS.foo_quota = -99
|
||||
self.assertEqual(-1, res.default)
|
||||
@ -94,6 +104,7 @@ class TestTrackedResource(testlib_api.SqlTestCase):
|
||||
session.add(item)
|
||||
|
||||
def setUp(self):
|
||||
cfg.CONF.set_override('quota_driver', QUOTA_DRIVER, group='QUOTAS')
|
||||
super(TestTrackedResource, self).setUp()
|
||||
self.setup_coreplugin(DB_PLUGIN_KLASS)
|
||||
self.resource = 'meh'
|
||||
|
||||
@ -160,6 +160,7 @@ class TestAuxiliaryFunctions(base.DietTestCase):
|
||||
'TrackedResource.mark_dirty') as mock_mark_dirty:
|
||||
self.registry.set_tracked_resource('meh', test_quota.MehModel)
|
||||
self.registry.register_resource_by_name('meh')
|
||||
self.registry.resources['meh']._track_resource_events = True
|
||||
res = self.registry.get_resource('meh')
|
||||
# This ensures dirty is true
|
||||
res._dirty_tenants.add('tenant_id')
|
||||
|
||||
@ -0,0 +1,11 @@
|
||||
---
|
||||
features:
|
||||
- |
|
||||
A new quota driver is added: ``DbQuotaNoLockDriver``. This driver, unlike
|
||||
``DbQuotaDriver``, does not create a unique lock per (resource,
|
||||
project_id). That may lead to a database deadlock state if the number of
|
||||
server requests exceeds the number of resolved resource creations, as
|
||||
described in `LP#1926787 <https://bugs.launchpad.net/neutron/+bug/1926787>`_.
|
||||
This driver relays on the database transactionality isolation and counts
|
||||
the number of used and reserved resources and, if available, creates the
|
||||
new resource reservations in one single database transaction.
|
||||
Loading…
Reference in New Issue
Block a user