nova/releasenotes/notes/libvirt-event-based-device-detach-23ac037004d753b1.yaml
Balazs Gibizer e56cc4f439 Replace blind retry with libvirt event waiting in detach
Nova so far applied a retry loop that tried to periodically detach the
device from libvirt while the device was visible in the domain xml. This
could lead to an issue where an already progressing detach on the
libvirt side is interrupted by nova re-sending the detach request for
the same device. See bug #1882521 for more information.

Also if there was both a persistent and a live domain the nova tried the
detach from both at the same call. This lead to confusion about the
result when such call failed. Was the detach failed partially?

We can do better, at least for the live detach case. Based on the
libvirt developers detaching from the persistent domain always
succeeds and it is a synchronous process. Detaching from the live
domain can be both synchronous or asynchronous depending on the guest
OS and the load on the hypervisor. But for live detach libvirt always
sends an event [1] nova can wait for.

So this patch does two things.

1) Separates the detach from the persistent domain from the detach from
   the live domain to make the error cases clearer.

2) Changes the retry mechanism.

   Detaching from the persistent domain is not retried. If libvirt
   reports device not found, while both persistent and live detach
   is needed, the error is ignored, and the process continues with
   the live detach. In any other case the error considered as fatal.

   Detaching from the live domain is changed to always wait for the
   libvirt event. In case of timeout, the live detach is retried.
   But a failure event from libvirt considered fatal, based on the
   information from the libvirt developers, so in this case the
   detach is not retried.

Related-Bug: #1882521

[1]https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectDomainEventDeviceRemovedCallback

Change-Id: I7f2b6330decb92e2838aa7cee47fb228f00f47da
2021-04-18 10:24:08 +02:00

12 lines
487 B
YAML

---
fixes:
- |
To fix `device detach issues`__ in the libvirt driver the detach logic has
been changed from a sleep based retry loop to waiting for libvirt domain
events. During this change we also introduced two new config options to
allow fine tuning the retry logic. For details see the description of the
new ``[libvirt]device_detach_attempts`` and
``[libvirt]device_detach_timeout`` config options.
.. __: https://bugs.launchpad.net/nova/+bug/1882521