openstack/nova-specs
Code Issues Proposed changes

Merge "Support initial allocation ratios"

Browse Source
This commit is contained in:
Zuul 2018-10-25 15:56:44 +00:00 committed by Gerrit Code Review
commit 4d2fbee4f9
1 changed files with 313 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

313
specs/stein/approved/initial-allocation-ratios.rst Normal file
View File

@ -0,0 +1,313 @@
..
This work is licensed under a Creative Commons Attribution 3.0 Unported
License.
http://creativecommons.org/licenses/by/3.0/legalcode
======================================
Default allocation ratio configuration
======================================
https://blueprints.launchpad.net/nova/+spec/initial-allocation-ratios
Provide separate CONF options for specifying the initial allocation
ratio for compute nodes. Change the default values for
CONF.xxx_allocation_ratio options to None and change the behaviour of
the resource tracker to only override allocation ratios for *existing*
compute nodes if the CONF.xxx_allocation_ratio value is not None.
The primary goal of this feature is to support both the API and config way to
pass allocation ratios.
Problem description
===================
Manually set placement allocation ratios are overwritten
--------------------------------------------------------------------
There is currently no way for an admin to set the allocation ratio on an
individual compute node resource provider's inventory record in the placement
API without the resource tracker eventually overwriting that value the next
time it runs the ``update_available_resources`` periodic task on the
``nova-compute`` service.
The saga of the allocation ratio values on the compute host
-----------------------------------------------------------
The process by which nova determines the allocation ratio for CPU, RAM and disk
resources on a hypervisor is confusing and `error`_ `prone`_. The
``compute_nodes`` table in the nova cell DB contains three fields representing
the allocation ratio for CPU, RAM and disk resources on that hypervisor. These
fields are populated using different default values depending on the version of
nova running on the ``nova-compute`` service.
.. _error: https://bugs.launchpad.net/nova/+bug/1742747
.. _prone: https://bugs.launchpad.net/nova/+bug/1789654
Upon starting up, the resource tracker in the ``nova-compute`` service worker
`checks`_ to see if a record exists in the ``compute_nodes`` table of the nova
cell DB for itself. If it does not find one, the resource tracker `creates`_ a
record in the table, `setting`_ the associated allocation ratio values in the
``compute_nodes`` table to the value it finds in the ``cpu_allocation_ratio``,
``ram_allocation_ratio`` and ``disk_allocation_ratio`` nova.conf configuration
options but only if the config option value is not equal to 0.0.
.. _checks: https://github.com/openstack/nova/blob/852de1e/nova/compute/resource_tracker.py#L566
.. _creates: https://github.com/openstack/nova/blob/852de1e/nova/compute/resource_tracker.py#L577-L590
.. _setting: https://github.com/openstack/nova/blob/6a68f9140/nova/compute/resource_tracker.py#L621-L645
The default values of the ``cpu_allocation_ratio``, ``ram_allocation_ratio``
and ``disk_allocation_ratio`` CONF options is `currently set`_ to ``0.0``.
.. _currently set: https://github.com/openstack/nova/blob/852de1e/nova/conf/compute.py#L400
The resource tracker saves these default ``0.0`` values to the
``compute_nodes`` table when the resource tracker calls ``save()`` on the
compute node object. However, there is `code`_ in the
``ComputeNode._from_db_obj`` that, upon **reading** the record back from the
database on first save, changes the values from ``0.0`` to ``16.0``, ``1.5`` or
``1.0``.
.. _code: https://github.com/openstack/nova/blob/852de1e/nova/objects/compute_node.py#L177-L207
The ``ComputeNode`` object that was ``save()``'d by the resource tracker has
these new values for some period of time while the record in the
``compute_nodes`` table continues to have the wrong ``0.0`` values. When the
resource tracker runs its ``update_available_resource()`` next perioidic task,
the new ``16.0``/``1.5``/``1.0`` values are then saved to the compute nodes
table.
There is a `fix`_ for `bug/1789654`_, which is to not persist
zero allocation ratios in ResourceTracker to avoid initializing placement
allocation_ratio with 0.0 (due to the allocation ratio of 0.0 being multiplied
by the total amount in inventory, leading to 0 resources shown on the system).
.. _fix: https://review.openstack.org/#/c/598365/
.. _bug/1789654: https://bugs.launchpad.net/nova/+bug/1789654
Use Cases
---------
An administrator would like to set allocation ratios for individual resources
on a compute node via the placement API *without that value being overwritten*
by the compute node's resource tracker.
An administrator chooses to only use the configuration file to set allocation
ratio overrides on their compute nodes and does not want to use the placement
API to set these ratios.
Proposed change
===============
First, we propose to change the default option values of existing
``CONF.cpu_allocation_ratio``, ``CONF.ram_allocation_ratio`` and
``CONF.disk_allocation_ratio`` options relating to allocation ratios to
``None`` from the existing default values of ``0.0``. The reason we change
it is that this value will be change from ``0.0`` to ``16.0``, ``1.5`` or
``1.0`` later, which is weird and confusing.
We will also change the resource tracker to **only** overwrite the compute
node's allocation ratios to the value of the ``cpu_allocation_ratio``,
``ram_allocation_ratio`` and ``disk_allocation_ratio`` CONF options **if the
value of these options is NOT ``None``**.
In other words, if any of these CONF options is set to something *other than*
``None``, then the CONF option should be considered the complete override value
for that resource class' allocation ratio. Even if an admin manually adjusts
the allocation ratio of the resource class in the placement API, the next time
the ``update_available_resource()`` periodic task runs, it will be overwritten
to the value of the CONF option.
Second, we propose to add 3 new nova.conf configuration options:
* ``initial_cpu_allocation_ratio``
* ``initial_ram_allocation_ratio``
* ``initial_disk_allocation_ratio``
That will used to determine how to set the *initial* allocation ratio of
``VCPU``, ``MEMORY_MB`` and ``DISK_GB`` resource classes when a compute worker
first starts up and creates its compute node record in the nova cell DB and
corresponding inventory records in the placement service. The value of these
new configuration options will only be used if the compute service's resource
tracker is not able to find a record in the placement service for the compute
node the resource tracker is managing.
The default value of each of these CONF options shall be ``16.0``, ``1.5``, and
``1.0`` respectively. This is to match the default values for the original
allocation ratio CONF options before they were set to ``0.0``.
These new ``initial_xxx_allocation_ratio`` CONF options shall **ONLY** be used
if the resource tracker detects no existing record in the ``compute_nodes``
nova cell DB for that hypervisor.
Finally, we will need also add an online data migration and continue to read
the ``xxx_allocation_ratio`` or ``initial_xxx_allocation_ratio`` config on
read from the DB if the values are ``0.0`` or ``None``. If it's an existing
record with 0.0 values, we'd want to do what the compute does, which is use
the configure ``xxx_allocation_ratio`` config if it's not None, and fallback
to using the ``initial_xxx_allocation_ratio`` otherwise.
And add an online data migration that updates all compute_nodes
table records that have ``0.0`` or ``None`` allocation ratios. Then we drop
that at some point with a blocker migration and remove the code in the
``nova.objects.ComputeNode._from_db_obj`` that adjusts allocation ratios.
We propose to add a nova-status upgrade check to iterate the cells looking
for compute_nodes records with ``0.0`` or ``None`` allocation ratios and signal
that as a warning that you haven't done the online data migration. We could
also check the conf options to see if they are explicitly set to 0.0 and if
so, we should fail the status check.
Alternatives
------------
None
Data model impact
-----------------
None
REST API impact
---------------
None
Security impact
---------------
None
Notifications impact
--------------------
None
Other end user impact
---------------------
None
Performance Impact
------------------
None
Other deployer impact
---------------------
None
Developer impact
----------------
None
Upgrade impact
--------------
We need an online data migrations for any compute_nodes with existing ``0.0``
and ``None`` allocation ratio. If it's an existing record with 0.0 values, we
will replace it with the configure ``xxx_allocation_ratio`` config if it's not
None, and fallback to using the ``initial_xxx_allocation_ratio`` otherwise.
.. note:: Migrating 0.0 allocation ratios from existing ``compute_nodes`` table
records is necessary because the ComputeNode object based on those table
records is what gets used in the scheduler [1]_, specifically the
``NUMATopologyFilter`` and ``CPUWeigher`` (the ``CoreFilter``,
``DiskFilter`` and ``RamFilter`` also use them but those filters are
deprecated for removal so they are not a concern here).
And clearly in order to take advantage of the ability to manually set
allocation ratios on a compute node, that hypervisor would need to be upgraded.
No impact to old compute hosts.
Implementation
==============
Assignee(s)
-----------
Primary assignee:
yikun
Work Items
----------
* Change the default values for ``CONF.xxx_allocation_ratio`` options to
``None``.
* Modify resource tracker to only set allocation ratios on the compute node
object when the CONF options are non- ``None``
* Add new ``initial_xxx_allocation_ratio`` CONF options and modify resource
tracker's initial compute node creation to use these values
* Remove code in the ``ComputeNode._from_db_obj()`` that changes allocation
ratio values
* Add a db online migration to process all compute_nodes with existing ``0.0``
and ``None`` allocation ratio.
* Add a nova-status upgrade check for ``0.0`` or ``None`` allocation ratio.
Dependencies
============
None
Testing
=======
No extraordinary testing outside normal unit and functional testing
Documentation Impact
====================
A release note explaining the use of the new ``initial_xxx_allocation_ratio``
CONF options should be created along with a more detailed doc in the admin
guide explaining the following primary scenarios:
* When the deployer wants to **ALWAYS** set an override value for a resource on
a compute node. This is where the deployer would ensure that the
``cpu_allocation_ratio``, ``ram_allocation_ratio`` and
``disk_allocation_ratio`` CONF options were set to a non- ``None`` value.
* When the deployer wants to set an **INITIAL** value for a compute node's
allocation ratio but wants to allow an admin to adjust this afterwards
without making any CONF file changes. This scenario uses the new
``initial_xxx_allocation_ratios`` for the initial ratio values and then shows
the deployer using the osc placement commands to manually set an allocation
ratio for a resource class on a resource provider.
* When the deployer wants to **ALWAYS** use the placement API to set allocation
ratios, then the deployer should ensure that ``CONF.xxx_allocation_ratio``
options are all set to ``None`` and the deployer should issue Placement
REST API calls to
``PUT /resource_providers/{uuid}/inventories/{resource_class}`` [2]_ or
``PUT /resource_providers/{uuid}/inventories`` [3]_ to set the allocation
ratios of their resources as needed (or use the related ``osc-placement``
plugin commands [4]_).
References
==========
.. [1] https://github.com/openstack/nova/blob/a534ccc5a7/nova/scheduler/host_manager.py#L255
.. [2] https://developer.openstack.org/api-ref/placement/#update-resource-provider-inventory
.. [3] https://developer.openstack.org/api-ref/placement/#update-resource-provider-inventories
.. [4] https://docs.openstack.org/osc-placement/latest/
Nova Stein PTG discussion:
* https://etherpad.openstack.org/p/nova-ptg-stein
Bugs:
* https://bugs.launchpad.net/nova/+bug/1742747
* https://bugs.launchpad.net/nova/+bug/1729621
* https://bugs.launchpad.net/nova/+bug/1739349
* https://bugs.launchpad.net/nova/+bug/1789654
History
=======
.. list-table:: Revisions
:header-rows: 1
* - Release Name
- Description
* - Stein
- Proposed