diff --git a/doc/source/admin/cells.rst b/doc/source/admin/cells.rst index bb83e967f352..d7e6b893a168 100644 --- a/doc/source/admin/cells.rst +++ b/doc/source/admin/cells.rst @@ -755,7 +755,13 @@ them far over quota when the unreachable cell returns. counting of quota usage from the placement service and API database to make quota usage calculations resilient to down or poor-performing cells in a multi-cell environment. See the :doc:`quotas documentation - ` for more details. + ` for more details. + + Starting in the 2023.2 Bobcat (28.0.0) release, it is possible to configure + unified limits quotas, which stores quota limits as Keystone unified limits + and counts quota usage from the placement service and API database. See the + :doc:`unified limits documentation ` for more + details. Performance of listing instances ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index 8cb5bf7156d1..79c6e9a21c67 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -126,7 +126,7 @@ Once you have an OpenStack deployment up and running, you will want to manage it. The below guides cover everything from creating initial flavor and image to log management and live migration of instances. -* :doc:`Quotas `: Managing project quotas in nova. +* :doc:`Quotas `: Managing project quotas in nova. * :doc:`Scheduling `: How the scheduler is configured, and how that will impact where compute instances land in your @@ -158,7 +158,7 @@ log management and live migration of instances. config-drive image-caching metadata-service - quotas + unified-limits networking security-groups security diff --git a/doc/source/admin/quotas.rst b/doc/source/admin/quotas.rst index c8000b3ba28c..8dce1ce48b33 100644 --- a/doc/source/admin/quotas.rst +++ b/doc/source/admin/quotas.rst @@ -1,7 +1,17 @@ +:orphan: + ============= Manage quotas ============= +.. warning:: + + As of Nova release 28.0.0 (2023.2 Bobcat), the ``nova.quota.DbQuotaDriver`` + has been deprecated and the default quota driver configuration will be + changed to the ``nova.quota.UnifiedLimitsDriver`` in the 29.0.0. (2024.1 + Caracal) release. See the :doc:`unified limits documentation + `. + .. note:: This section provides deployment information about the quota feature. For diff --git a/doc/source/admin/unified-limits.rst b/doc/source/admin/unified-limits.rst new file mode 100644 index 000000000000..dbbebe66ff2a --- /dev/null +++ b/doc/source/admin/unified-limits.rst @@ -0,0 +1,195 @@ +============================ +Manage Unified Limits Quotas +============================ + +.. note:: + + This section provides deployment information about the quota feature. For + end-user information about quotas, including information about the type of + quotas available, refer to the :doc:`user guide `. + +Since the Nova 28.0.0 (2023.2 Bobcat) release, it is recommended to use +`Keystone unified limits`_ for Nova quota limits. + +For information about legacy quota limits, see the :doc:`legacy quota +documentation `. + +To enable unified limits quotas, set the quota driver in the Nova +configuration: + +.. code-block:: ini + + [quota] + driver = nova.quota.UnifiedLimitsDriver + +Quota limits are set and retrieved for enforcement using the `Keystone API`_. + +.. _Keystone unified limits: https://docs.openstack.org/keystone/latest/admin/unified-limits.html +.. _Keystone API: https://docs.openstack.org/api-ref/identity/v3/index.html#unified-limits + +To prevent system capacities from being exhausted without notification, you can +set up quotas. Quotas are operational limits. For example, the number of +servers allowed for each project can be controlled so that cloud resources +are optimized. Quotas can be enforced at both the global (default) and the +project level. + +Resource usage is counted from the placement service with unified limits. + + +Checking quota +-------------- + +When calculating limits for a given resource and project, the following checks +are made in order: + +#. Project-specific limits + + Depending on the resource, is there a project-specific limit on the resource + in Keystone limits? If so, use that as the limit. You can create these + resource limits using: + + .. code-block:: console + + $ openstack limit create --service nova --project --resource-limit 5 servers + +#. Default limits + + Use the Keystone registered limit for the resource as the limit. You can + create these default limits using: + + .. code-block:: console + + $ openstack registered limit create --service nova --default-limit 5 servers + + +Rechecking quota +~~~~~~~~~~~~~~~~ + +If :oslo.config:option:`quota.recheck_quota` is True (this is the default), +Nova will perform a second quota check after allocating resources. The first +quota check is performed before resources are allocated. Rechecking quota +ensures that quota limits are strictly enforced and prevents any possibility of +resource allocation going over the quota limit in the event of racing parallel +API requests. + +It can be disabled by setting :oslo.config:option:`quota.recheck_quota` to +False if strict quota enforcement is not important to the operator. + + +Quota usage from placement +-------------------------- + +With unified limits quotas, it is required that quota resource usage is counted +from placement. As such, the +:oslo.config:option:`quota.count_usage_from_placement` configuration option is +ignored when :oslo.config:option:`quota.driver` is set to +``nova.quota.UnifiedLimitsDriver``. + +There are some things to note when quota resource usage is counted from +placement: + +* Counted usage will not be accurate in an environment where multiple Nova + deployments are sharing a placement deployment because currently placement + has no way of partitioning resource providers between different Nova + deployments. Operators who are running multiple Nova deployments that share a + placement deployment should not use the ``nova.quota.UnifiedLimitsDriver``. + +* Behavior will be different for resizes. During a resize, resource allocations + are held on both the source and destination (even on the same host, see + https://bugs.launchpad.net/nova/+bug/1790204) until the resize is confirmed + or reverted. Quota usage will be inflated for servers in this state. + +* The ``populate_queued_for_delete`` and ``populate_user_id`` online data + migrations must be completed before usage can be counted from placement. + Until the data migration is complete, the system will fall back to legacy + quota usage counting from cell databases depending on the result of an EXISTS + database query during each quota check. Use + ``nova-manage db online_data_migrations`` to run online data migrations. + +* Behavior will be different for unscheduled servers in ``ERROR`` state. A + server in ``ERROR`` state that has never been scheduled to a compute host + will not have placement allocations, so it will not consume quota usage for + cores and ram. + +* Behavior will be different for servers in ``SHELVED_OFFLOADED`` state. A + server in ``SHELVED_OFFLOADED`` state will not have placement allocations, so + it will not consume quota usage for cores and ram. Note that because of this, + it will be possible for a request to unshelve a server to be rejected if the + user does not have enough quota available to support the cores and ram needed + by the server to be unshelved. + + +Configuration +------------- + +View and update default quota values +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To list all default quotas for a project, run: + +.. code-block:: console + + $ openstack registered limit list + +.. note:: + + This lists default quotas for all services and not just nova. + +To create a default quota limit, run: + +.. code-block:: console + + $ openstack registered limit create --service nova --default-limit + +.. note:: + + Creating or updating registered limits requires a system-scoped + authorization token by default. See the `Keystone tokens documentation`_ for + more information. + + .. _Keystone tokens documentation: https://docs.openstack.org/keystone/latest/admin/tokens-overview.html#operation_create_system_token + +To update a default value, run: + +.. code-block:: console + + $ openstack registered limit set --default-limit + +View and update quota values for a project +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To list quotas for a project, run: + +.. code-block:: console + + $ openstack limit list --project + +.. note:: + + This lists project quotas for all services and not just nova. + +To list quotas for all projects, you must have a system-scoped authorization +token and run: + +.. code-block:: console + + $ openstack limit list + +To update quotas for a project, run: + +.. code-block:: console + + $ openstack limit set --resource-limit + + +Migration to unified limits quotas +---------------------------------- + +There is a `nova-manage limits migrate_to_unified_limits`_ command available +to help with moving from legacy Nova database quotas to Keystone unified limits +quotas. The command will read quota limits from the Nova database and call the +Keystone API to create the corresponding unified limits. Per-user quota limits +will **not** be copied into Keystone because per-user quotas are not supported +in unified limits. + +.. _nova-manage limits migrate_to_unified_limits: https://docs.openstack.org/nova/latest/cli/nova-manage.html#limits-migrate-to-unified-limits diff --git a/doc/source/index.rst b/doc/source/index.rst index 8cd5ae9ceb17..a213fa530a24 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -180,7 +180,7 @@ Once you are running nova, the following information is extremely useful. * :doc:`Flavors `: What flavors are and why they are used. * :doc:`Upgrades `: How nova is designed to be upgraded for minimal service impact, and the order you should do them in. -* :doc:`Quotas `: Managing project quotas in nova. +* :doc:`Quotas `: Managing project quotas in nova. * :doc:`Aggregates `: Aggregates are a useful way of grouping hosts together for scheduling purposes. * :doc:`Scheduling `: How the scheduler is @@ -200,7 +200,7 @@ Once you are running nova, the following information is extremely useful. admin/index user/flavors admin/upgrades - user/quotas + user/unified-limits admin/vendordata Reference Material diff --git a/doc/source/user/quotas.rst b/doc/source/user/quotas.rst index 7377e0f1a242..76b663a9bd64 100644 --- a/doc/source/user/quotas.rst +++ b/doc/source/user/quotas.rst @@ -1,7 +1,17 @@ +:orphan: + ====== Quotas ====== +.. warning:: + + As of Nova release 28.0.0 (2023.2 Bobcat), the ``nova.quota.DbQuotaDriver`` + has been deprecated and the default quota driver configuration will be + changed to the ``nova.quota.UnifiedLimitsDriver`` in the 29.0.0 (2024.1 + Caracal) release. See the :doc:`unified limits documentation + `. + Nova uses a quota system for setting limits on resources such as number of instances or amount of CPU that a specific project or user can use. diff --git a/doc/source/user/unified-limits.rst b/doc/source/user/unified-limits.rst new file mode 100644 index 000000000000..af090d821402 --- /dev/null +++ b/doc/source/user/unified-limits.rst @@ -0,0 +1,143 @@ +===================== +Unified Limits Quotas +===================== + +Since the Nova 28.0.0 (2023.2 Bobcat) release, it is recommended to use +`Keystone unified limits`_ for Nova quota limits. + +For information about legacy quota limits, see the :doc:`legacy quota +documentation `. + +Nova uses a quota system for setting limits on resources such as number of +instances or amount of CPU that a specific project or user can use. + +Quota limits are set by admin and retrieved for enforcement using the `Keystone +API`_. + +.. _Keystone unified limits: https://docs.openstack.org/keystone/latest/admin/unified-limits.html +.. _Keystone API: https://docs.openstack.org/api-ref/identity/v3/index.html#unified-limits + +Types of quota +-------------- + +.. list-table:: + :header-rows: 1 + :widths: 10 40 + + * - Quota name + - Description + * - class:VCPU + - Number of instance cores (VCPUs) allowed per project. + * - servers + - Number of instances allowed per project. + * - server_key_pairs + - Number of key pairs allowed per user. + * - server_metadata_items + - Number of metadata items allowed per instance. + * - class:MEMORY_MB + - Megabytes of instance ram allowed per project. + * - server_groups + - Number of server groups per project. + * - server_group_members + - Number of servers per server group. + * - class:DISK_GB + - Gigabytes of instance disk allowed per project. + * - class: + - Any resource in the placement service that is allocated by Nova can have + a quota limit specified for it. Example: class:VGPU. + +The following quotas were previously available but were removed in microversion +2.36 as they proxied information available from the networking service. + +.. list-table:: + :header-rows: 1 + :widths: 10 40 + + * - Quota name + - Description + * - fixed_ips + - Number of fixed IP addresses allowed per project. This number + must be equal to or greater than the number of allowed + instances. + * - floating_ips + - Number of floating IP addresses allowed per project. + * - networks + - Number of networks allowed per project (no longer used). + * - security_groups + - Number of security groups per project. + * - security_group_rules + - Number of security group rules per project. + +Similarly, the following quotas were previously available but were removed in +microversion 2.57 as the personality files feature was deprecated. + +.. list-table:: + :header-rows: 1 + :widths: 10 40 + + * - Quota name + - Description + * - injected_files + - Number of injected files allowed per project. + * - injected_file_content_bytes + - Number of content bytes allowed per injected file. + * - injected_file_path_bytes + - Length of injected file path. + + +Usage +----- + +Project quotas +~~~~~~~~~~~~~~ + +To list all default quotas for projects, run: + +.. code-block:: console + + $ openstack registered limit list + +.. note:: + + This lists default quotas for all services and not just nova. + +For example: + +.. code-block:: console + + $ openstack registered limit list + +----------------------------------+----------------------------------+------------------------------------+---------------+-------------+-----------+ + | ID | Service ID | Resource Name | Default Limit | Description | Region ID | + +----------------------------------+----------------------------------+------------------------------------+---------------+-------------+-----------+ + | eeee406035fc4a7892c6fc6fcc76e61a | b5d97619e6354c03be50c6b1589e6547 | image_size_total | 1000 | None | RegionOne | + | b99d35b1b7b74cd0a26a6311963328af | b5d97619e6354c03be50c6b1589e6547 | image_stage_total | 1000 | None | RegionOne | + | 70945789dad24082bc2a8cdbf53a5091 | b5d97619e6354c03be50c6b1589e6547 | image_count_total | 100 | None | RegionOne | + | 908d86c0ac3f46d0bb45ed9194710ea7 | b5d97619e6354c03be50c6b1589e6547 | image_count_uploading | 100 | None | RegionOne | + | be6dfeebb7c340e8b93b602d41fbff9b | 8b22bf8a66fa4524a522b2a21865bbf2 | servers | 10 | None | None | + | 8a658096236549788e61f4fcbd5a4a12 | 8b22bf8a66fa4524a522b2a21865bbf2 | class:VCPU | 20 | None | None | + | 63890db7d6a14401ba55e7f7022b95d0 | 8b22bf8a66fa4524a522b2a21865bbf2 | class:MEMORY_MB | 51200 | None | None | + | 221ba1c19d2c4272952663828d659013 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_metadata_items | 128 | None | None | + | 8e61cabd2e854a11bdd2cf94efd702d1 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_injected_files | 5 | None | None | + | 3d259390f70e4e9b88ecb2b0fa075f9b | 8b22bf8a66fa4524a522b2a21865bbf2 | server_injected_file_content_bytes | 10240 | None | None | + | a12e10b991cc4bdd8e6ff30f6e6c15ac | 8b22bf8a66fa4524a522b2a21865bbf2 | server_injected_file_path_bytes | 255 | None | None | + | a32a9080be6b4a5481c16a91fe329e6f | 8b22bf8a66fa4524a522b2a21865bbf2 | server_key_pairs | 100 | None | None | + | 86408bb7a0e542b18404ec7d348da820 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_groups | 10 | None | None | + | 17c4552c5aad4afca4813f37530fc897 | 8b22bf8a66fa4524a522b2a21865bbf2 | server_group_members | 10 | None | None | + +----------------------------------+----------------------------------+------------------------------------+---------------+-------------+-----------+ + +To list the currently set quota values for your project, run: + +.. code-block:: console + + $ openstack limit list + +For example: + +.. code-block:: console + + $ openstack limit list + +----------------------------------+----------------------------------+----------------------------------+---------------+----------------+-------------+-----------+ + | ID | Project ID | Service ID | Resource Name | Resource Limit | Description | Region ID | + +----------------------------------+----------------------------------+----------------------------------+---------------+----------------+-------------+-----------+ + | 8b3364b2241e4090aaaa49355c7a5b56 | 5cd3281595a9497ba87209701cd9f3f2 | 8b22bf8a66fa4524a522b2a21865bbf2 | class:VCPU | 5 | None | None | + +----------------------------------+----------------------------------+----------------------------------+---------------+----------------+-------------+-----------+ diff --git a/nova/conf/quota.py b/nova/conf/quota.py index e5b4b8dc7381..4742249c3816 100644 --- a/nova/conf/quota.py +++ b/nova/conf/quota.py @@ -175,14 +175,23 @@ Possible values: cfg.StrOpt('driver', default='nova.quota.DbQuotaDriver', choices=[ - ('nova.quota.DbQuotaDriver', 'Stores quota limit information ' - 'in the database and relies on the ``quota_*`` configuration ' - 'options for default quota limit values. Counts quota usage ' - 'on-demand.'), + ('nova.quota.DbQuotaDriver', '(deprecated) Stores quota limit ' + 'information in the database and relies on the ``quota_*`` ' + 'configuration options for default quota limit values. Counts ' + 'quota usage on-demand.'), ('nova.quota.NoopQuotaDriver', 'Ignores quota and treats all ' 'resources as unlimited.'), - ('nova.quota.UnifiedLimitsDriver', 'Do not use. Still being ' - 'developed.') + ('nova.quota.UnifiedLimitsDriver', 'Uses Keystone unified limits ' + 'to store quota limit information and relies on resource ' + 'usage counting from Placement. Counts quota usage on-demand. ' + 'Resources missing unified limits in Keystone will be treated ' + 'as a quota limit of 0, so it is important to ensure all ' + 'resources have registered limits in Keystone. The ``nova-manage ' + 'limits migrate_to_unified_limits`` command can be used to copy ' + 'existing quota limits from the Nova database to Keystone ' + 'unified limits via the Keystone API. Alternatively, unified ' + 'limits can be created manually using the OpenStackClient or ' + 'by calling the Keystone API directly.'), ], help=""" Provides abstraction for quota checks. Users can configure a specific diff --git a/releasenotes/notes/deprecate-legacy-quota-driver-56906ca0e2842839.yaml b/releasenotes/notes/deprecate-legacy-quota-driver-56906ca0e2842839.yaml new file mode 100644 index 000000000000..1213cebdf9f3 --- /dev/null +++ b/releasenotes/notes/deprecate-legacy-quota-driver-56906ca0e2842839.yaml @@ -0,0 +1,5 @@ +deprecations: + - | + The ``nova.quota.DbQuotaDriver`` is marked as deprecated and the default + quota driver configuration is planned to be changed to the + ``nova.quota.UnifiedLimitsDriver`` in the 29.0.0 (2024.1 Caracal) release.