nova/nova/virt/libvirt
Balazs Gibizer 30317e6b3f Replace blind retry with libvirt event waiting in detach
Nova so far applied a retry loop that tried to periodically detach the
device from libvirt while the device was visible in the domain xml. This
could lead to an issue where an already progressing detach on the
libvirt side is interrupted by nova re-sending the detach request for
the same device. See bug #1882521 for more information.

Also if there was both a persistent and a live domain the nova tried the
detach from both at the same call. This lead to confusion about the
result when such call failed. Was the detach failed partially?

We can do better, at least for the live detach case. Based on the
libvirt developers detaching from the persistent domain always
succeeds and it is a synchronous process. Detaching from the live
domain can be both synchronous or asynchronous depending on the guest
OS and the load on the hypervisor. But for live detach libvirt always
sends an event [1] nova can wait for.

So this patch does two things.

1) Separates the detach from the persistent domain from the detach from
   the live domain to make the error cases clearer.

2) Changes the retry mechanism.

   Detaching from the persistent domain is not retried. If libvirt
   reports device not found, while both persistent and live detach
   is needed, the error is ignored, and the process continues with
   the live detach. In any other case the error considered as fatal.

   Detaching from the live domain is changed to always wait for the
   libvirt event. In case of timeout, the live detach is retried.
   But a failure event from libvirt considered fatal, based on the
   information from the libvirt developers, so in this case the
   detach is not retried.

Related-Bug: #1882521

[1]https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectDomainEventDeviceRemovedCallback

Change-Id: I7f2b6330decb92e2838aa7cee47fb228f00f47da
(cherry picked from commit e56cc4f439)
2021-05-05 09:47:35 +02:00
..
storage Remove six.text_type (1/2) 2020-12-13 11:25:31 +00:00
volume libvirt: Drop support for Xen 2021-01-22 10:06:40 +00:00
__init__.py Remove vi modelines 2014-02-03 14:19:44 +00:00
blockinfo.py libvirt: Drop support for Xen 2021-01-22 10:06:40 +00:00
config.py Merge "libvirt: Add guest generation for vDPA" 2021-03-16 00:03:00 +00:00
designer.py libvirt: Add guest generation for vDPA 2021-03-12 11:48:40 +00:00
driver.py Replace blind retry with libvirt event waiting in detach 2021-05-05 09:47:35 +02:00
event.py libvirt: Define and emit DeviceRemovedEvent and DeviceRemovalFailedEvent 2021-02-22 11:05:15 +01:00
guest.py Replace blind retry with libvirt event waiting in detach 2021-05-05 09:47:35 +02:00
host.py [Arm64][libvirt] Fix read bytes from json files 2021-03-24 12:09:13 +00:00
imagebackend.py libvirt: Remove 'hypervisor_version' from 'libvirt_info' 2021-01-22 12:02:15 +00:00
imagecache.py Include removal of ephemeral backing files in the image cache manager 2020-10-09 12:24:06 +01:00
instancejobtracker.py Remove translation of log messages 2017-06-13 11:20:28 +07:00
machine_type_utils.py nova-manage: Add libvirt list_unset_machine_type command 2021-03-03 14:03:49 +00:00
migration.py libvirt: Remove MIN_LIBVIRT_FILE_BACKED_DISCARD_VERSION 2020-09-22 10:29:37 +01:00
utils.py Merge "Refactor ResourceRequest constructor" 2021-02-07 06:36:58 +00:00
vif.py libvirt: Add guest generation for vDPA 2021-03-12 11:48:40 +00:00