diff --git a/doc/source/admin/aggregates.rst b/doc/source/admin/aggregates.rst index 353ebefec65a..0cc6053ee849 100644 --- a/doc/source/admin/aggregates.rst +++ b/doc/source/admin/aggregates.rst @@ -355,6 +355,38 @@ configuration: - :oslo.config:option:`filter_scheduler.aggregate_image_properties_isolation_separator` - :oslo.config:option:`filter_scheduler.aggregate_image_properties_isolation_namespace` +.. _image-caching-aggregates: + +Image Caching +------------- + +Aggregates can be used as a way to target multiple compute nodes for the purpose of +requesting that images be pre-cached for performance reasons. + +.. note:: + + Some of the virt drivers provide image caching support, which improves performance + of second-and-later boots of the same image by keeping the base image in an on-disk + cache. This avoids the need to re-download the image from Glance, which reduces + network utilization and time-to-boot latency. Image pre-caching is the act of priming + that cache with images ahead of time to improve performance of the first boot. + +Assuming an aggregate called ``my-aggregate`` where two images should +be pre-cached, running the following command will initiate the +request: + +.. code-block:: console + + $ nova aggregate-cache-images my-aggregate image1 image2 + +Note that image pre-caching happens asynchronously in a best-effort +manner. The images and aggregate provided are checked by the server +when the command is run, but the compute nodes are not checked to see +if they support image caching until the process runs. Progress and +results are logged by each compute, and the process sends +``aggregate.cache_images.start`` and ``aggregate.cache_images.end`` +notifications, which may be useful for monitoring the operation +externally. References ---------- diff --git a/doc/source/admin/image-caching.rst b/doc/source/admin/image-caching.rst new file mode 100644 index 000000000000..44bcfe1a6e0b --- /dev/null +++ b/doc/source/admin/image-caching.rst @@ -0,0 +1,77 @@ +============= +Image Caching +============= + +Nova supports caching base images on compute nodes when using a +supported virt driver. As of the Ussuri release, that list includes +``hyperv``, ``libvirt``, and ``vmwareapi``. + +What is Image Caching? +---------------------- + +In order to understand what image caching is and why it is beneficial, +it helps to be familiar with the process by which an instance is +booted from a given base image. When a new instance is created on a +compute node, the following general steps are performed by the compute +manager in conjunction with the virt driver: + +#. Download the base image from glance +#. Copy or COW the base image to create a new root disk image for the instance +#. Boot the instance using the new root disk image + +The first step involves downloading the entire base image to the local +disk on the compute node, which could involve many gigabytes of +network traffic, storage, and many minutes of latency between the +start of the boot process and actually running the instance. When the +virt driver supports image caching, step #1 above may be skipped if +the base image is already present on the compute node. This is most +often the case when another instance has been booted on that node from +the same base image recently. If present, the download operation can +be skipped, which greatly reduces the time-to-boot for the second and +subsequent instances that use the same base image, as well as avoids +load on the glance server and the network connection. + +By default, the compute node will periodically scan the images it has +cached, looking for base images that are not used by any instances on +the node that are older than a configured lifetime (24 hours by +default). Those unused images are deleted from the cache directory +until they are needed again. + +For more information about configuring image cache behavior, see the +documentation for the following configuration options: + +- :oslo.config:option:`image_cache_subdirectory_name` +- :oslo.config:option:`image_cache_manager_interval` +- :oslo.config:option:`remove_unused_base_images` +- :oslo.config:option:`remove_unused_original_minimum_age_seconds` +- :oslo.config:option:`image_cache.precache_concurrency` + +.. note:: + + Some ephemeral backend drivers may not use or need image caching, + or may not behave in the same way as others. For example, when + using the ``rbd`` backend with the ``libvirt`` driver and a shared + pool with glance, images are COW'd at the storage level and thus + need not be downloaded (and thus cached) at the compute node at + all. + +Image pre-caching +----------------- + +It may be beneficial to pre-cache images on compute nodes in order to +achieve low time-to-boot latency for new instances immediately. This +is often useful when rolling out a new version of an application where +downtime is important and having the new images already available on +the compute nodes is critical. + +Nova provides (since the Ussuri release) a mechanism to request that +images be cached without having to boot an actual instance on a +node. This best-effort service operates at the host aggregate level in +order to provide an efficient way to indicate that a large number of +computes should receive a given set of images. If the computes that +should pre-cache an image are not already in a defined host aggregate, +that must be done first. + +For information on how to perform aggregate-based image pre-caching, +see the :ref:`image-caching-aggregates` section of the Host aggregates +documentation. diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index 799bf3cd4d8c..0b87ff157f0b 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -32,6 +32,7 @@ operating system, and exposes functionality over a web-based API. evacuate.rst flavors.rst huge-pages.rst + image-caching.rst live-migration-usage.rst manage-logs.rst manage-the-cloud.rst