Add spec for expose-auto-converge-post-copy
Change-Id: I08ab3fb105fd83a0b3095ea30a0f0518cfaa24ea
This commit is contained in:
ya.wang
261
specs/train/approved/expose-auto-converge-post-copy.rst
Normal file
261
specs/train/approved/expose-auto-converge-post-copy.rst
Normal file
@@ -0,0 +1,261 @@
|
|||||||
|
..
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||||
|
License.
|
||||||
|
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
==================================
|
||||||
|
Expose auto converge and post copy
|
||||||
|
==================================
|
||||||
|
|
||||||
|
https://blueprints.launchpad.net/nova/+spec/expose-auto-converge-post-copy
|
||||||
|
|
||||||
|
Problem description
|
||||||
|
===================
|
||||||
|
|
||||||
|
Currently auto converge and post copy can only be enabled/disabled via
|
||||||
|
configuration, which is somewhat inflexible. If an application sensitive to
|
||||||
|
reduced performance (some scientific computing applications may be more
|
||||||
|
sensitive to memory access latency) is on a host with these options enabled,
|
||||||
|
live migration may cause the application to raise an error. Therefore, the user
|
||||||
|
wants to control whether to enable/disable auto converge or post copy during
|
||||||
|
live migration.
|
||||||
|
|
||||||
|
Use Cases
|
||||||
|
---------
|
||||||
|
|
||||||
|
* Some applications do not want increased risk of being rebooted due to a
|
||||||
|
network failure or memory page access failure during post-copy
|
||||||
|
live-migration.
|
||||||
|
|
||||||
|
* Some applications are performance sensitive (such as some scientific
|
||||||
|
computing applications); such applications do not want performance throttled
|
||||||
|
back by the auto-converge feature during live-migration.
|
||||||
|
|
||||||
|
* Some applications would like to avoid reboot risk and performance
|
||||||
|
throttling. If the network between two compute nodes is interrupted during
|
||||||
|
post-copy live-migration, the live-migration will fail and the user will need
|
||||||
|
to reset the instance to make it available. Therefore such applications do
|
||||||
|
not want use both features during live-migration.
|
||||||
|
|
||||||
|
* For the above problems, the operator wants to control whether a single
|
||||||
|
instance enables auto converge or post copy during live migration. But
|
||||||
|
currently the minimum unit that can be controlled is the compute node.
|
||||||
|
|
||||||
|
Proposed change
|
||||||
|
===============
|
||||||
|
|
||||||
|
Support for auto converge and post copy requires QEMU version >= 2.5.0. Since
|
||||||
|
the Rocky release, the minimum required version of QEMU is 2.5.0 [1]_.
|
||||||
|
Therefore, all compute nodes using the libvirt driver should support these
|
||||||
|
features. There are flags from the libvirt ``virDomainMigrateFlags`` enum
|
||||||
|
[2]_::
|
||||||
|
|
||||||
|
...
|
||||||
|
VIR_MIGRATE_AUTO_CONVERGE = 8192
|
||||||
|
VIR_MIGRATE_POSTCOPY = 32768
|
||||||
|
...
|
||||||
|
|
||||||
|
The configurations ``live_migration_permit_auto_converge`` and
|
||||||
|
``live_migration_permit_post_copy`` can only affect the hypervisor by
|
||||||
|
modifying the configuration, but traits can affect a single instance.
|
||||||
|
|
||||||
|
In order to request the feature (scheduling an instance to nodes that provide
|
||||||
|
the feature) we propose defining two new traits. The traits are reported by the
|
||||||
|
libvirt driver, regardless of the conf:
|
||||||
|
|
||||||
|
* ``COMPUTE_MIGRATE_AUTO_CONVERGE``
|
||||||
|
* ``COMPUTE_MIGRATE_POST_COPY``
|
||||||
|
|
||||||
|
Introduce two new flavor extra specs:
|
||||||
|
|
||||||
|
* ``compute:live_migration_auto_converge=true/false``
|
||||||
|
* ``compute:live_migration_post_copy=true/false``
|
||||||
|
|
||||||
|
And introduce two new image properties:
|
||||||
|
|
||||||
|
* ``compute_live_migration_auto_converge=true/false``
|
||||||
|
* ``compute_live_migration_post_copy=true/false``
|
||||||
|
|
||||||
|
Use these properties, instead of asking the operator to set
|
||||||
|
``required``/``forbidden`` on the traits. Before calling placement, when
|
||||||
|
``compute:live_migration_auto_converge=true`` or
|
||||||
|
``compute:live_migration_post_copy=true``, we add required traits
|
||||||
|
for the corresponding feature to the placement request. When
|
||||||
|
``compute:live_migration_auto_converge=false`` and
|
||||||
|
``compute:live_migration_post_copy=false``, we just add nothing to
|
||||||
|
the placement request. Thus we still can schedule an instance on a host with
|
||||||
|
the features but we disable these two features for that instance. We use these
|
||||||
|
keys in the scheduler to optionally add required traits to ensure that the
|
||||||
|
instance can land on a host that is capable of the requested behavior. The
|
||||||
|
libvirt driver will then interpret the values to decide whether to use the
|
||||||
|
features during live migration. For example, if the flavor says "false":
|
||||||
|
|
||||||
|
* We don't add the trait to the scheduling request, so the instance can land
|
||||||
|
anywhere.
|
||||||
|
* The driver will **not** use the feature for live-migrate, regardless of what
|
||||||
|
the compute's config says.
|
||||||
|
|
||||||
|
By default, when the operator creates an instance without any related metadata,
|
||||||
|
the scheduler will not care whether the host supports auto-converge or
|
||||||
|
post-copy. If the configurations ``live_migration_permit_auto_converge`` or
|
||||||
|
``live_migration_permit_post_copy`` are True, the libvirt driver will prefer to
|
||||||
|
use auto-converge or post-copy. These can be used when the operator wants **all
|
||||||
|
instances** on a given compute node to use auto-converge/post-copy. For
|
||||||
|
example:
|
||||||
|
|
||||||
|
* If an instance that has not requested related metadata is scheduled to a host
|
||||||
|
that enabled ``live_migration_permit_auto_converge`` or
|
||||||
|
``live_migration_permit_post_copy``, then libvirt will try to use
|
||||||
|
auto-converge or post-copy during live migration.
|
||||||
|
|
||||||
|
If the operator creates instance with
|
||||||
|
``compute:live_migration_auto_converge`=true/false`` or
|
||||||
|
``compute:live_migration_post_copy=true/false``,
|
||||||
|
these metadata will override the configurations:
|
||||||
|
``live_migration_permit_auto_converge`` or
|
||||||
|
``live_migration_permit_post_copy``.
|
||||||
|
|
||||||
|
When ``compute:live_migration_auto_converge`` and
|
||||||
|
``compute_live_migration_post_copy`` are both true or flavor extra specs
|
||||||
|
is in conflict with image properties, the 'create' API call will raise an
|
||||||
|
exception.
|
||||||
|
|
||||||
|
When using auto-converge during live migration, if the operator calls the force
|
||||||
|
complete API, libvirt will not be converted to use post-copy because it's not
|
||||||
|
required in flavor extra specs or image properties.
|
||||||
|
|
||||||
|
According to this spec [3]_, if post-copy is enabled during live migration, the
|
||||||
|
abort API call will be rejected by libvirt driver. Now we can reject the
|
||||||
|
request in the API by checking ``hw_live_migration_permit_reboot_risk``
|
||||||
|
properties.
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
Another method is to use traits in flavor extra_specs/image properties. This
|
||||||
|
could work well when the operators need auto-converge/post-copy. But it can't
|
||||||
|
be used to disable auto-converge/post-copy.
|
||||||
|
Since the Rocky release, all libvirt hypervisor hosts support
|
||||||
|
auto-converge/post-copy, which means every libvirt hypervisor host would have
|
||||||
|
traits ``COMPUTE_MIGRATE_AUTO_CONVERGE`` and ``COMPUTE_MIGRATE_POST_COPY``.
|
||||||
|
If operators want to not use auto-converge or post-copy, they would use
|
||||||
|
forbidden traits: ``traits:COMPUTE_MIGRATE_AUTO_CONVERGE=forbidden`` or
|
||||||
|
``traits:COMPUTE_MIGRATE_POST_COPY=forbidden``. Which means **don't** schedule
|
||||||
|
my vm to the hosts who support auto-converge/post-copy, as the above says, this
|
||||||
|
means that all libvirt compute nodes will be ignored. The result will be that
|
||||||
|
the vm creation failed because the compute node can't be scheduled.
|
||||||
|
|
||||||
|
Data model impact
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
Add the two image properties to the ImageMeta object:
|
||||||
|
|
||||||
|
* compute_live_migration_auto_converge
|
||||||
|
* compute_live_migration_post_copy
|
||||||
|
|
||||||
|
The ImageMeta is stored in table instance_system_metadata, no schema
|
||||||
|
modification is needed.
|
||||||
|
|
||||||
|
REST API impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Security impact
|
||||||
|
---------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Notifications impact
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other end user impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Performance Impact
|
||||||
|
------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Other deployer impact
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Developer impact
|
||||||
|
----------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Upgrade impact
|
||||||
|
--------------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
Ya Wang
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
* Support for new placement traits.
|
||||||
|
|
||||||
|
* Libvirt driver changes to report traits to placement, the traits will be
|
||||||
|
reported by the libvirt driver as part of ``update_provider_tree``. This will
|
||||||
|
*not* be added to the generic compute capabilities dict inherited by all the
|
||||||
|
virt drivers because these traits are libvirt-specific.
|
||||||
|
|
||||||
|
* Scheduler changes to translate metadata to traits.
|
||||||
|
|
||||||
|
* Recalculate ``_live_migration_flags`` before live migration start in
|
||||||
|
the libvirt driver.
|
||||||
|
|
||||||
|
* Add functional tests and unit tests.
|
||||||
|
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
|
||||||
|
Testing
|
||||||
|
=======
|
||||||
|
|
||||||
|
Unit tests and functional tests will be included to test the new functionality.
|
||||||
|
|
||||||
|
|
||||||
|
Documentation Impact
|
||||||
|
====================
|
||||||
|
|
||||||
|
* The live migration document should be changed to introduce this new feature.
|
||||||
|
|
||||||
|
References
|
||||||
|
==========
|
||||||
|
|
||||||
|
.. [1] https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix
|
||||||
|
.. [2] https://libvirt.org/html/libvirt-libvirt-domain.html#virDomainMigrateFlags
|
||||||
|
.. [3] https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/auto-live-migration-completion.html
|
||||||
|
|
||||||
|
|
||||||
|
History
|
||||||
|
=======
|
||||||
|
|
||||||
|
.. list-table:: Revisions
|
||||||
|
:header-rows: 1
|
||||||
|
|
||||||
|
* - Release Name
|
||||||
|
- Description
|
||||||
|
* - Train
|
||||||
|
- Introduced
|
||||||
Reference in New Issue
Block a user