openstack/nova
Code Issues Proposed changes

Add documentation for unified limits

Browse Source 
This adds documentation for unified limits and signals deprecation of
the nova.quota.DbQuotaDriver.

Related to blueprint unified-limits-nova-tool-and-docs

Change-Id: I3951317111396aa4df36c5700b4d4dd33e721a74
This commit is contained in:
melanie witt 2023-08-30 00:31:09 +00:00
parent 395501c876
commit 8f0817f078
9 changed files with 389 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
doc/source/admin/cells.rst
View File

@ -755,7 +755,13 @@ them far over quota when the unreachable cell returns.
counting of quota usage from the placement service and API database counting of quota usage from the placement service and API database
to make quota usage calculations resilient to down or poor-performing to make quota usage calculations resilient to down or poor-performing
cells in a multi-cell environment. See the :doc:`quotas documentation cells in a multi-cell environment. See the :doc:`quotas documentation
</user/quotas>` for more details. </admin/quotas>` for more details.
Starting in the 2023.2 Bobcat (28.0.0) release, it is possible to configure
unified limits quotas, which stores quota limits as Keystone unified limits
and counts quota usage from the placement service and API database. See the
:doc:`unified limits documentation </admin/unified-limits>` for more
details.
Performance of listing instances Performance of listing instances
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

4
doc/source/admin/index.rst
View File

@ -126,7 +126,7 @@ Once you have an OpenStack deployment up and running, you will want to manage
it. The below guides cover everything from creating initial flavor and image to it. The below guides cover everything from creating initial flavor and image to
log management and live migration of instances. log management and live migration of instances.
* :doc:`Quotas </admin/quotas>`: Managing project quotas in nova. * :doc:`Quotas </admin/unified-limits>`: Managing project quotas in nova.
* :doc:`Scheduling </admin/scheduling>`: How the scheduler is * :doc:`Scheduling </admin/scheduling>`: How the scheduler is
configured, and how that will impact where compute instances land in your configured, and how that will impact where compute instances land in your
@ -158,7 +158,7 @@ log management and live migration of instances.
config-drive config-drive
image-caching image-caching
metadata-service metadata-service
quotas unified-limits
networking networking
security-groups security-groups
security security

10
doc/source/admin/quotas.rst
View File

@ -1,7 +1,17 @@
:orphan:
============= =============
Manage quotas Manage quotas
============= =============
.. warning::
As of Nova release 28.0.0 (2023.2 Bobcat), the ``nova.quota.DbQuotaDriver``
has been deprecated and the default quota driver configuration will be
changed to the ``nova.quota.UnifiedLimitsDriver`` in the 29.0.0. (2024.1
Caracal) release. See the :doc:`unified limits documentation
</admin/unified-limits>`.
.. note:: .. note::
This section provides deployment information about the quota feature. For This section provides deployment information about the quota feature. For

195
doc/source/admin/unified-limits.rst Normal file
View File

@ -0,0 +1,195 @@
============================
Manage Unified Limits Quotas
============================
.. note::
This section provides deployment information about the quota feature. For
end-user information about quotas, including information about the type of
quotas available, refer to the :doc:`user guide </user/unified-limits>`.
Since the Nova 28.0.0 (2023.2 Bobcat) release, it is recommended to use
`Keystone unified limits`_ for Nova quota limits.
For information about legacy quota limits, see the :doc:`legacy quota
documentation </admin/quotas>`.
To enable unified limits quotas, set the quota driver in the Nova
configuration:
.. code-block:: ini
[quota]
driver = nova.quota.UnifiedLimitsDriver
Quota limits are set and retrieved for enforcement using the `Keystone API`_.
.. _Keystone unified limits: https://docs.openstack.org/keystone/latest/admin/unified-limits.html
.. _Keystone API: https://docs.openstack.org/api-ref/identity/v3/index.html#unified-limits
To prevent system capacities from being exhausted without notification, you can
set up quotas. Quotas are operational limits. For example, the number of
servers allowed for each project can be controlled so that cloud resources
are optimized. Quotas can be enforced at both the global (default) and the
project level.
Resource usage is counted from the placement service with unified limits.
Checking quota
--------------
When calculating limits for a given resource and project, the following checks
are made in order:
#. Project-specific limits
Depending on the resource, is there a project-specific limit on the resource
in Keystone limits? If so, use that as the limit. You can create these
resource limits using:
.. code-block:: console
$ openstack limit create --service nova --project <project> --resource-limit 5 servers
#. Default limits
Use the Keystone registered limit for the resource as the limit. You can
create these default limits using:
.. code-block:: console
$ openstack registered limit create --service nova --default-limit 5 servers
Rechecking quota
~~~~~~~~~~~~~~~~
If :oslo.config:option:`quota.recheck_quota` is True (this is the default),
Nova will perform a second quota check after allocating resources. The first
quota check is performed before resources are allocated. Rechecking quota
ensures that quota limits are strictly enforced and prevents any possibility of
resource allocation going over the quota limit in the event of racing parallel
API requests.
It can be disabled by setting :oslo.config:option:`quota.recheck_quota` to
False if strict quota enforcement is not important to the operator.
Quota usage from placement
--------------------------
With unified limits quotas, it is required that quota resource usage is counted
from placement. As such, the
:oslo.config:option:`quota.count_usage_from_placement` configuration option is
ignored when :oslo.config:option:`quota.driver` is set to
``nova.quota.UnifiedLimitsDriver``.
There are some things to note when quota resource usage is counted from
placement:
* Counted usage will not be accurate in an environment where multiple Nova
deployments are sharing a placement deployment because currently placement
has no way of partitioning resource providers between different Nova
deployments. Operators who are running multiple Nova deployments that share a
placement deployment should not use the ``nova.quota.UnifiedLimitsDriver``.
* Behavior will be different for resizes. During a resize, resource allocations
are held on both the source and destination (even on the same host, see
https://bugs.launchpad.net/nova/+bug/1790204) until the resize is confirmed
or reverted. Quota usage will be inflated for servers in this state.
* The ``populate_queued_for_delete`` and ``populate_user_id`` online data
migrations must be completed before usage can be counted from placement.
Until the data migration is complete, the system will fall back to legacy
quota usage counting from cell databases depending on the result of an EXISTS
database query during each quota check. Use
``nova-manage db online_data_migrations`` to run online data migrations.
* Behavior will be different for unscheduled servers in ``ERROR`` state. A
server in ``ERROR`` state that has never been scheduled to a compute host
will not have placement allocations, so it will not consume quota usage for
cores and ram.
* Behavior will be different for servers in ``SHELVED_OFFLOADED`` state. A
server in ``SHELVED_OFFLOADED`` state will not have placement allocations, so
it will not consume quota usage for cores and ram. Note that because of this,
it will be possible for a request to unshelve a server to be rejected if the
user does not have enough quota available to support the cores and ram needed
by the server to be unshelved.
Configuration
-------------
View and update default quota values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To list all default quotas for a project, run:
.. code-block:: console
$ openstack registered limit list
.. note::
This lists default quotas for all services and not just nova.
To create a default quota limit, run:
.. code-block:: console
$ openstack registered limit create --service nova --default-limit <value> <resource-name>
.. note::
Creating or updating registered limits requires a system-scoped
authorization token by default. See the `Keystone tokens documentation`_ for
more information.
.. _Keystone tokens documentation: https://docs.openstack.org/keystone/latest/admin/tokens-overview.html#operation_create_system_token
To update a default value, run:
.. code-block:: console
$ openstack registered limit set --default-limit <value> <registered-limit-id>
View and update quota values for a project
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To list quotas for a project, run:
.. code-block:: console
$ openstack limit list --project <project>
.. note::
This lists project quotas for all services and not just nova.
To list quotas for all projects, you must have a system-scoped authorization
token and run:
.. code-block:: console
$ openstack limit list
To update quotas for a project, run:
.. code-block:: console
$ openstack limit set --resource-limit <value> <limit-id>
Migration to unified limits quotas
----------------------------------
There is a `nova-manage limits migrate_to_unified_limits`_ command available
to help with moving from legacy Nova database quotas to Keystone unified limits
quotas. The command will read quota limits from the Nova database and call the
Keystone API to create the corresponding unified limits. Per-user quota limits
will **not** be copied into Keystone because per-user quotas are not supported
in unified limits.
.. _nova-manage limits migrate_to_unified_limits: https://docs.openstack.org/nova/latest/cli/nova-manage.html#limits-migrate-to-unified-limits

4
doc/source/index.rst
View File

@ -180,7 +180,7 @@ Once you are running nova, the following information is extremely useful.
* :doc:`Flavors </user/flavors>`: What flavors are and why they are used. * :doc:`Flavors </user/flavors>`: What flavors are and why they are used.
* :doc:`Upgrades </admin/upgrades>`: How nova is designed to be upgraded for minimal * :doc:`Upgrades </admin/upgrades>`: How nova is designed to be upgraded for minimal
service impact, and the order you should do them in. service impact, and the order you should do them in.
* :doc:`Quotas </user/quotas>`: Managing project quotas in nova. * :doc:`Quotas </user/unified-limits>`: Managing project quotas in nova.
* :doc:`Aggregates </admin/aggregates>`: Aggregates are a useful way of grouping * :doc:`Aggregates </admin/aggregates>`: Aggregates are a useful way of grouping
hosts together for scheduling purposes. hosts together for scheduling purposes.
* :doc:`Scheduling </admin/scheduling>`: How the scheduler is * :doc:`Scheduling </admin/scheduling>`: How the scheduler is
@ -200,7 +200,7 @@ Once you are running nova, the following information is extremely useful.
admin/index admin/index
user/flavors user/flavors
admin/upgrades admin/upgrades
user/quotas user/unified-limits
admin/vendordata admin/vendordata
Reference Material Reference Material

10
doc/source/user/quotas.rst
View File

@ -1,7 +1,17 @@
:orphan:
====== ======
Quotas Quotas
====== ======
.. warning::
As of Nova release 28.0.0 (2023.2 Bobcat), the ``nova.quota.DbQuotaDriver``
has been deprecated and the default quota driver configuration will be
changed to the ``nova.quota.UnifiedLimitsDriver`` in the 29.0.0 (2024.1
Caracal) release. See the :doc:`unified limits documentation
</user/unified-limits>`.
Nova uses a quota system for setting limits on resources such as number of Nova uses a quota system for setting limits on resources such as number of
instances or amount of CPU that a specific project or user can use. instances or amount of CPU that a specific project or user can use.

143
doc/source/user/unified-limits.rst Normal file
View File

@ -0,0 +1,143 @@
=====================
Unified Limits Quotas
=====================
Since the Nova 28.0.0 (2023.2 Bobcat) release, it is recommended to use
`Keystone unified limits`_ for Nova quota limits.
For information about legacy quota limits, see the :doc:`legacy quota
documentation </user/quotas>`.
Nova uses a quota system for setting limits on resources such as number of
instances or amount of CPU that a specific project or user can use.
Quota limits are set by admin and retrieved for enforcement using the `Keystone
API`_.
.. _Keystone unified limits: https://docs.openstack.org/keystone/latest/admin/unified-limits.html
.. _Keystone API: https://docs.openstack.org/api-ref/identity/v3/index.html#unified-limits
Types of quota
--------------
.. list-table::
:header-rows: 1
:widths: 10 40
* - Quota name
- Description
* - class:VCPU
- Number of instance cores (VCPUs) allowed per project.
* - servers
- Number of instances allowed per project.
* - server_key_pairs
- Number of key pairs allowed per user.
* - server_metadata_items
- Number of metadata items allowed per instance.
* - class:MEMORY_MB
- Megabytes of instance ram allowed per project.
* - server_groups
- Number of server groups per project.
* - server_group_members
- Number of servers per server group.
* - class:DISK_GB
- Gigabytes of instance disk allowed per project.
* - class:<any resource in the placement service>
- Any resource in the placement service that is allocated by Nova can have
a quota limit specified for it. Example: class:VGPU.
The following quotas were previously available but were removed in microversion
2.36 as they proxied information available from the networking service.
.. list-table::
:header-rows: 1
:widths: 10 40
* - Quota name
- Description
* - fixed_ips
- Number of fixed IP addresses allowed per project. This number
must be equal to or greater than the number of allowed
instances.
* - floating_ips
- Number of floating IP addresses allowed per project.
* - networks
- Number of networks allowed per project (no longer used).
* - security_groups
- Number of security groups per project.
* - security_group_rules
- Number of security group rules per project.
Similarly, the following quotas were previously available but were removed in
microversion 2.57 as the personality files feature was deprecated.
.. list-table::
:header-rows: 1
:widths: 10 40
* - Quota name
- Description
* - injected_files
- Number of injected files allowed per project.
* - injected_file_content_bytes
- Number of content bytes allowed per injected file.
* - injected_file_path_bytes
- Length of injected file path.
Usage
-----
Project quotas
~~~~~~~~~~~~~~
To list all default quotas for projects, run:
.. code-block:: console
$ openstack registered limit list
.. note::
This lists default quotas for all services and not just nova.
For example:
.. code-block:: console
$ openstack registered limit list
+----------------------------------+----------------------------------+------------------------------------+---------------+-------------+-----------+
| ID | Service ID | Resource Name | Default Limit | Description | Region ID |
+----------------------------------+----------------------------------+------------------------------------+---------------+-------------+-----------+
| eeee406035fc4a7892c6fc6fcc76e61a | b5d97619e6354c03be50c6b1589e6547 | image_size_total | 1000 | None | RegionOne |
| b99d35b1b7b74cd0a26a6311963328af | b5d97619e6354c03be50c6b1589e6547 | image_stage_total | 1000 | None | RegionOne |
| 70945789dad24082bc2a8cdbf53a5091 | b5d97619e6354c03be50c6b1589e6547 | image_count_total | 100 | None | RegionOne |
| 908d86c0ac3f46d0bb45ed9194710ea7 | b5d97619e6354c03be50c6b1589e6547 | image_count_uploading | 100 | None | RegionOne |
| be6dfeebb7c340e8b93b602d41fbff9b | 8b22bf8a66fa4524a522b2a21865bbf2 | servers | 10 | None | None |
| 8a658096236549788e61f4fcbd5a4a12 | 8b22bf8a66fa4524a522b2a21865bbf2 | class:VCPU | 20 | None | None |
| 63890db7d6a14401ba55e7f7022b95d0 | 8b22bf8a66fa4524a522b2a21865bbf2 | class:MEMORY_MB | 51200 | None | None |
| 221ba1c19d2c4272952663828d659013 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_metadata_items | 128 | None | None |
| 8e61cabd2e854a11bdd2cf94efd702d1 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_injected_files | 5 | None | None |
| 3d259390f70e4e9b88ecb2b0fa075f9b | 8b22bf8a66fa4524a522b2a21865bbf2 | server_injected_file_content_bytes | 10240 | None | None |
| a12e10b991cc4bdd8e6ff30f6e6c15ac | 8b22bf8a66fa4524a522b2a21865bbf2 | server_injected_file_path_bytes | 255 | None | None |
| a32a9080be6b4a5481c16a91fe329e6f | 8b22bf8a66fa4524a522b2a21865bbf2 | server_key_pairs | 100 | None | None |
| 86408bb7a0e542b18404ec7d348da820 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_groups | 10 | None | None |
| 17c4552c5aad4afca4813f37530fc897 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_group_members | 10 | None | None |
+----------------------------------+----------------------------------+------------------------------------+---------------+-------------+-----------+
To list the currently set quota values for your project, run:
.. code-block:: console
$ openstack limit list
For example:
.. code-block:: console
$ openstack limit list
+----------------------------------+----------------------------------+----------------------------------+---------------+----------------+-------------+-----------+
| ID | Project ID | Service ID | Resource Name | Resource Limit | Description | Region ID |
+----------------------------------+----------------------------------+----------------------------------+---------------+----------------+-------------+-----------+
| 8b3364b2241e4090aaaa49355c7a5b56 | 5cd3281595a9497ba87209701cd9f3f2 | 8b22bf8a66fa4524a522b2a21865bbf2 | class:VCPU | 5 | None | None |
+----------------------------------+----------------------------------+----------------------------------+---------------+----------------+-------------+-----------+

21
nova/conf/quota.py
View File

@ -175,14 +175,23 @@ Possible values:
cfg.StrOpt('driver', cfg.StrOpt('driver',
default='nova.quota.DbQuotaDriver', default='nova.quota.DbQuotaDriver',
choices=[ choices=[
('nova.quota.DbQuotaDriver', 'Stores quota limit information ' ('nova.quota.DbQuotaDriver', '(deprecated) Stores quota limit '
'in the database and relies on the ``quota_*`` configuration ' 'information in the database and relies on the ``quota_*`` '
'options for default quota limit values. Counts quota usage ' 'configuration options for default quota limit values. Counts '
'on-demand.'), 'quota usage on-demand.'),
('nova.quota.NoopQuotaDriver', 'Ignores quota and treats all ' ('nova.quota.NoopQuotaDriver', 'Ignores quota and treats all '
'resources as unlimited.'), 'resources as unlimited.'),
('nova.quota.UnifiedLimitsDriver', 'Do not use. Still being ' ('nova.quota.UnifiedLimitsDriver', 'Uses Keystone unified limits '
'developed.') 'to store quota limit information and relies on resource '
'usage counting from Placement. Counts quota usage on-demand. '
'Resources missing unified limits in Keystone will be treated '
'as a quota limit of 0, so it is important to ensure all '
'resources have registered limits in Keystone. The ``nova-manage '
'limits migrate_to_unified_limits`` command can be used to copy '
'existing quota limits from the Nova database to Keystone '
'unified limits via the Keystone API. Alternatively, unified '
'limits can be created manually using the OpenStackClient or '
'by calling the Keystone API directly.'),
], ],
help=""" help="""
Provides abstraction for quota checks. Users can configure a specific Provides abstraction for quota checks. Users can configure a specific

5
releasenotes/notes/deprecate-legacy-quota-driver-56906ca0e2842839.yaml Normal file
View File

@ -0,0 +1,5 @@
deprecations:
- |
The ``nova.quota.DbQuotaDriver`` is marked as deprecated and the default
quota driver configuration is planned to be changed to the
``nova.quota.UnifiedLimitsDriver`` in the 29.0.0 (2024.1 Caracal) release.