Commit Graph

23 Commits (740ab628f4d060f7caa33d9c3dc1e189255468c1)

Author SHA1 Message Date
Brian Haley 7594bb0627 Remove the dependency on the "mock" package
Now that we are python3 only, we should move to using the built
in version of mock that supports all of our testing needs and
remove the dependency on the "mock" package.

This patch moves all references to "import mock" to
"from unittest import mock". It also cleans up some new line

Fixed an inconsistency in the OVSBridge.deferred() definition
as it needs to also have an *args argument.

Fixed an issue where an l3-agent test was mocking
functools.partial, causing a python3.8 failure.

Unit tests only, removing from tests/ affects
functional tests which need additional work.

Change-Id: I40e8a8410840c3774c72ae1a8054574445d66ece
3 years ago
Boden R 6fe35c8039 use get reader/writer session from neutron-lib
The get_reader_session and get_writer_session functions are available
in neutron-lib. This patch consumes them by removing the functions
from neutron and using lib's instead.


Change-Id: I4055172799a0d9a0ceb387b6d47e421e1c270ada
5 years ago
Ihar Hrachyshka 07bfe6adb9 CountableResource: try count/get functions for all plugins
It's of no guarantee that core plugin implements counter/getter function
for a CountableResource. Instead of just trying core plugin, try every
plugin registered in the directory.

To retain backwards compatibility, we also make sure that core plugin is
checked first.

Change-Id: I5245e217e1f44281f85febbdfaf873321253dc5d
Closes-Bug: #1714769
6 years ago
Sergey Belous a8109af65f Extend Quota API to report usage statistics
Extend existing quota api to report a quota set. The quota set
will contain a set of resources and its corresponding reservation,
limits and in_use count for each tenant.

DocImpact:Documentation describing the new API as well as the new
information that it exposes.

Co-Authored-By: Prince Boateng<>
Change-Id: Ief2a6a4d2d7085e2a9dcd901123bc4fe6ac7ca22
Related-bug: #1599488
6 years ago
Ihar Hrachyshka 8a0faef877 TestTrackedResource: register core plugin in directory
This is achieved by switching the test class to BaseTestCase and using
setup_coreplugin. This makes later calls to neutron.objects.db.api
functions that extract the plugin from plugins directory to work.

The test class registers some sqlalchemy events for models not
registered in BASEV2, which later makes unregister_events fail because
of failure to compare those model classes with other registered model
classes (those that belong to proper model classes like Port). Before
the patch, the core plugin was not correctly registered in plugins
directory, and no code was actually loading the plugin, so no other
sqla events were in _REGISTERED_SQLA_EVENTS, which explains why
previously calls to unregister_events hasn't failed.

Good news is that the new parent class does clean up all events itself,
so we don't need to set the cleanup hook ourselves.

This switch also facilitated removal of some other preparatory steps
from the class that are already implemented in the new parent.

Closes-Bug: #1677636
Change-Id: Id462e532066ac3cd5e2f76c3670da7f7820c6581
6 years ago
Manjeet Singh Bhatia 93d8376c44 OVO for Quotas and Reservation
This patch introduces and integrates Oslo-Versioned Object for
ResourceDelta, Reservation, Quota and QuotaUsage model classes.

Partially-Implements: blueprint adopt-oslo-versioned-objects-for-db
Co-Authored-By: Victor Morales <>
Change-Id: Ic058e66e6780e1f6ff25a5b2bbe7294959905765
6 years ago
Armando Migliaccio ca751a1486 Spin off context module

Partially-implements: blueprint neutron-lib

Change-Id: I48cf45dc1b07035d952152eac2548a3bd9fc2832
6 years ago
Ann Kamyshnikova 4f17f70089 Stop using legacy facade
Introduce get_reader_session() and get_writer_session()
and replace get_session() with them.
Mark get_session as depricated.

Stop using get_engine from legacy facade. Use writer engine for
places where it is required.

Partially-Implements blueprint: enginefacade-switch

Change-Id: I28b741bfa27bf04cbe273586e6e3e00e14fbe683
6 years ago
Kevin Benton b87715d764 Protect against '.delete()' for quota and revisions
The devref was suggesting to do exactly the wrong thing for cases where
we have SQLAlchemy event listeners waiting for deleted objects.
DELETE statements fail to trigger any kind of SQLAlchemy event
listeners and the majority of our delete operations are small
(<10 items at once) so the performance difference is negligible
(especially since we do it so infrequently).

Change-Id: Id142179418a7b94d8d9695871b3fcd5dcc64730c
6 years ago
Luong Anh Tuan 4f3be300b2 Replaces uuid.uuid4 with uuidutils.generate_uuid()
Openstack common has a wrapper for generating uuids.We should
use that function when generating uuids for consistency.

Change-Id: Id13e01ba59f37a5be90db1fe778e9735f0e8b9b6
Closes-Bug: #1082248
6 years ago
Jenkins fcc15554c5 Merge "Except if tracked resource registered as countable" 7 years ago
Kevin Benton 5d597367cf Except if tracked resource registered as countable
If a resource was registered in the quota engine before
being set as a tracked resource, it would have been registered
as a CountableResource instead of a TrackedResource.

This would cause a failure later when the resources were
being unregistered because the quota engine thought it
should have been a TrackedResource and tried to call
unregister_events on it.

This patch just prevents resources from being registered
as both types. It also callse unregister_all_resources as
part of the standard test case cleanup since the vast majority
of tests are using a plugin that may register resources.

Closes-Bug: #1612222
Change-Id: I8a40f38d7c0e5aeca257ba62115fa9b02ad5aa93
7 years ago
Henry Gessau 61cc14fd67 Switch to neutron-lib for model_base
Change-Id: If5b2b4cc0346515ddef3da1255ab49327c8e5732
7 years ago
Dariusz Smigiel df9411dc11 Rename DB columns: tenant -> project
All occurences of ``tenant_id`` across the database are renamed
to ``project_id``. Both options are equally valid, but ``project_id``
is preferred.
To inform external users about the change, HasTenant class was

Partially-Implements: blueprint keystone-v3

Change-Id: I87a8ef342ccea004731ba0192b23a8e79bc382dc
7 years ago
Bhagyashri Shewale 88e899f7a0 Fix module's import order
Made corrections in import order for built-in, third party and
project specific modules as per OpenStack import standards [1].


Change-Id: I899deefd6ee4732d6c0afd17a5afbe42b0fa37ba
7 years ago
Carl Baldwin bf2f1f1f87 Remove references to model_base through models_v2
I saw a patch set [1] that was running in to circular imports and
removing these.  I thought it'd be better to remove them all in one


Change-Id: If1a9ba3091ac99f01f73b98a7459358c3efd4dd5
7 years ago
Brandon Palm 2503dfb239 Fixed a bunch of typos throughout Neutron
Went through all of the docstrings in Neutron and did
some cleanup.  I'm sure there are bunch more that I have missed.

Change-Id: Ib29d2de1c580880c89ed4fd069e1515d0977a3e7
8 years ago
ajmiller 4a8c2b875e Reduce the chance of random check/gate test failures
As previously implemented, the TestTrackedResource class is designed
to inject random failures into the gate. It generates random numbers
within the range of 0..10000, and will fail if it generates duplicate
random numbers during its run.

This patch creates UUIDs instead of random numbers, and makes the
chance of an collision vanishingly small.

Change-Id: I0cf535d1c5a3995a50b506aafce10e983872dcb7
Closes-bug: #1494021
8 years ago
Salvatore Orlando 7da1724d44 Improve DB operations for quota reservation
This patch deals with the lock wait timeout and the deadlock errors
observed under high concurrency (api_workers >= 4) with the pymysql
driver. It includes the following changes:

- Stop setting dirty status for resource usage when creating
  reservation, as usage of reserved resources is not tracked anymore;
- Add a variable, increasing delay when retrying make_reservation
  upon a DBDeadlock error in order to reduce the chances of further
- Enable transaction retry upon DBDeadlock errors for set_quota_usage;
- Do not resync quota usage while making reservation. This puts a lot
  of stress on the database and is also wasteful since resource usage
  is very likely to change again once the transaction is committed;
- Use autonested_transaction to simplify logic around when the
  nested flag should be used.

Change-Id: I7a335f9ebea3c0d6fee6e6b757554e045a66075c
Closes-Bug: #1486134
Related-Blueprint: better-quotas
8 years ago
salvatore e10b008f7a Do not track active reservations
Reservations have a transient nature: a reservation lifespan
typically begins and ends with a single request.
Therefore tracking reserved amounts for each tenant and resource
is not nearly as efficient as tracking resource usage.
Indeed it is fairly easy to verify that the overhead for tracking
reserved amounts is much greater than the one needed for counting
active reservations for each tenant and resource.

This patch removes the logic for tracking reservations, and
replaces it with an explicit count of active reservations.

Please note that this patch does not adjust accordingly the
ResourceUsage DB model. This will be done in a separate patch with
an expand migration; this should avoid most merge conflicts before
the final patch for restoring reservation logic merges.

Related-Blueprint: better-quotas

Change-Id: Ib5e3bd61c1bc0fc8a5d612dae5c1740a8834a980
8 years ago
Salvatore Orlando 574b25b857 Reservations support
Add the concept of resource reservation in neutron.
Usage tracking logic is also updated to support reservations.
Reservations are not however available with the now deprecated
configuration-based quota driver.

The base API controller will now use reservations to perform
quota checks rather than counting resource usage and then
invoking the limit_check routine.

The limit_check routine however has not been removed and
depreacated as a part of this patch. In order to ensure all
quota drivers expose a consistent interface, a
make_reservation method has been added to the configuration
based driver as well. This method simply performs "old-style"
limit checks by counting resource usage and then invoking


Implements blueprint better-quotas.

Change-Id: Ifea07f461def564884af5b291c8a56655a4d818b
8 years ago
Salvatore Orlando b39e1469e8 Add plural names for quota resources
Introduce a 'plural_name' attribute in the BaseResource
class in the neutron.quota.resource module. This will
enable to specify the plural name of a resource when it's
entered in the quota registry.

This will come handy for managing reservations and also
removes the need for passing the collection name when
counting a resource.

Also, the name of the 'resources' parameter for the
_count_resource routine has ben changed to collection_name,
as the previous one was misleading, since it led to believe
it should be used for a collection, whereas it is simply
the name of a collection.

Change-Id: I0b61ba1856e9552cc14574be28fec6c77fb8783c
Related-Blueprint: better-quotas
8 years ago
Salvatore Orlando 3c584084c3 Introduce usage data tracking for Neutron
This patch introduces application logic support for tracking
Neutron resource usage data, thus introducing a different
way of enforcing quota limits, which now relies on records
tracking resource usage for each tenant.

When these records go "out-of-sync" with actual resource usage,
the quota usage table entry is marked "dirty".
And it will be resynchronized the next time a resource count is
requested. Changes in resource utilization are detected using
SQLAlchemy events.

This patch however does not automatically enable resource usage
tracking for any plugin. Plugins must explicitly declare for which
resources they wish to enable tracking. To this aim, this patch
provides the @tracked_resources decorator.

As some operators might wish to not use resource usage tracking, this
patch adds the track_quota_usage configuration option. If set to
False, DB-level usage tracking will not be performed, and
resources will be counted as before, ie: invoking a plugin method.

Usage tracking is performed using a new resource type,
TrackedResource, which can work side by side with CountableResource,
the one Neutron used so far. To this aim a ResourceRegistry class
has been introduced for registering and managing resources. Most
of the resource management code previously embedded in the
QuotaEngine class is being moved to ResourceRegistry as a part of
this patch.

Partially implements blueprint better-quota

Change-Id: If461399900973a38d7b82e0b3ac54fc052f09529
8 years ago