Add docs for the tripleo_nova_image_cache playbook
Provides some examples, tries to explain the interaction with nova, and mention the caveats when running with DCN. Change-Id: Ie32599e86afd336fb4fa31707fdcfe23690d035e
This commit is contained in:
Oliver Walsh
parent
bf90c44ccd
commit
bc759d1979
|
|
@ -18,6 +18,7 @@ post cloud deployment.
|
|||
validations/index
|
||||
|
||||
|
||||
|
||||
Post Cloud Deployment Advanced Topics
|
||||
=====================================
|
||||
|
||||
|
|
@ -32,3 +33,4 @@ or configured post cloud deployment.
|
|||
quiesce_compute
|
||||
updating_network_configuration_post_deployment
|
||||
vm_snapshot
|
||||
pre_cache_images
|
||||
|
|
|
|||
|
|
@ -0,0 +1,179 @@
|
|||
.. _precache_image:
|
||||
|
||||
Pre-caching images on Compute Nodes
|
||||
===================================
|
||||
|
||||
Fetching an image from Glance is often the most time consuming step when
|
||||
booting an instance. This can be particularly significant in cases where the
|
||||
data must traverse a high latency or limited bandwidth network, for example
|
||||
with :doc:`../features/distributed_compute_node`.
|
||||
|
||||
An ansible playbook is available to ensure images are already cached in cases
|
||||
where instance creation times must be minimized.
|
||||
|
||||
.. admonition:: Ussuri
|
||||
:class: ussuri
|
||||
|
||||
Since Ussuri Nova also provides an API to pre-cache images on Compute nodes.
|
||||
See the `Nova Image pre-caching documentation <https://docs.openstack.org/nova/ussuri/admin/image-caching.html#image-pre-caching>`_.
|
||||
|
||||
.. note:: The Nova Image Cache is not used when using Ceph RBD for Glange images and Nova ephemeral disk. See `Nova Image Caching documentation <https://docs.openstack.org/nova/ussuri/admin/image-caching.html>`_.
|
||||
|
||||
Image Cache Cleanup
|
||||
-------------------
|
||||
|
||||
The nova-compute service remains responsible for cleaning up old unused images
|
||||
on a compute node.
|
||||
A periodic job examines each of the images that are not currently used by an
|
||||
instance on the host.
|
||||
If an image is older than the configured maximum age it will be removed.
|
||||
|
||||
When an image is pre-cached the modification time is set to the current
|
||||
time. This effectively sets the image age back to 0.
|
||||
Therefore the pre-caching task should be repeated on an interval that is less
|
||||
than the maximum image age to ensure images remain cached.
|
||||
|
||||
Configuring the maximum image age
|
||||
---------------------------------
|
||||
|
||||
The default maximum image age is 86400 seconds (24 hours).
|
||||
This can be increased for all computes by setting `NovaImageCacheTTL` in the
|
||||
deployment parameters::
|
||||
|
||||
[stack@undercloud-0 ~]$ cat nova-cache-environment.yaml
|
||||
parameter_defaults:
|
||||
# Set the max image age to 30 days
|
||||
NovaImageCacheTTL: 2592000
|
||||
|
||||
Alternatively `NovaImageCacheTTL` can be set for individual compute roles::
|
||||
|
||||
[stack@undercloud-0 ~]$ cat nova-cache-environment.yaml
|
||||
parameter_defaults:
|
||||
# Set the max image age to 30 days for the ComputeSite1 role
|
||||
ComputeSite1Parameters:
|
||||
NovaImageCacheTTL: 2592000
|
||||
# Set the max image age to 7 days for the ComputeSite2 role
|
||||
ComputeSite2Parameters:
|
||||
NovaImageCacheTTL: 604800
|
||||
# Any other Compute roles default to 86400
|
||||
|
||||
.. _cache_all_computes:
|
||||
|
||||
Pre-caching a list of images on all Compute nodes
|
||||
-------------------------------------------------
|
||||
|
||||
Generate an ansible inventory for the stack name (default `overcloud`)::
|
||||
|
||||
[stack@undercloud-0 ~]$ . stackrc
|
||||
(undercloud) [stack@undercloud-0 ~]$ tripleo-ansible-inventory --plan overcloud --static-yaml-inventory inventory.yaml
|
||||
|
||||
Determine the list of image IDs to pre-cache::
|
||||
|
||||
[stack@undercloud-0 ~]$ . overcloudrc
|
||||
(overcloud) [stack@undercloud-0 ~]$ openstack image list
|
||||
+--------------------------------------+---------+--------+
|
||||
| ID | Name | Status |
|
||||
+--------------------------------------+---------+--------+
|
||||
| 07bc2424-753b-4f65-9da5-5a99d8383fe6 | image_0 | active |
|
||||
| d5187afa-c821-4f22-aa4b-4e76382bef86 | image_1 | active |
|
||||
+--------------------------------------+---------+--------+
|
||||
|
||||
Add the image ids to an argument file for the ansible playbook::
|
||||
|
||||
(overcloud) [stack@undercloud-0 ~]$ cat <<EOF > nova_cache_args.yml
|
||||
tripleo_nova_image_cache_images:
|
||||
- id: 07bc2424-753b-4f65-9da5-5a99d8383fe6
|
||||
- id: d5187afa-c821-4f22-aa4b-4e76382bef86
|
||||
EOF
|
||||
|
||||
Source the overcloud rc file to provide the necessary credentials for image download::
|
||||
|
||||
[stack@undercloud-0 ~]$ . overcloudrc
|
||||
|
||||
Run the `tripleo_nova_image_cache` playbook::
|
||||
|
||||
(overcloud) [stack@undercloud-0 ~]$ ansible-playbook -i inventory.yaml --extra-vars "@nova_cache_args.yml" /usr/share/ansible/tripleo-playbooks/tripleo_nova_image_cache.yml
|
||||
|
||||
PLAY [TripleO Nova image cache management] ***************************************************************************************************************************************************************************************************
|
||||
|
||||
TASK [tripleo-nova-image-cache : Cache image 07bc2424-753b-4f65-9da5-5a99d8383fe6] ***********************************************************************************************************************************************************
|
||||
changed: [compute-0]
|
||||
changed: [compute-1]
|
||||
|
||||
TASK [tripleo-nova-image-cache : Cache image d5187afa-c821-4f22-aa4b-4e76382bef86] ***********************************************************************************************************************************************************
|
||||
changed: [compute-0]
|
||||
changed: [compute-1]
|
||||
|
||||
.. note:: If the image already exists in cache then no change is reported however the image modification time is updated
|
||||
|
||||
.. warning:: The ansible `forks` config option (default=5) will affect the number of concurrent image downloads. Consider the load on the image service if adjusting this.
|
||||
|
||||
Multi-stacks inventory
|
||||
----------------------
|
||||
|
||||
When a multi-stack deployment is used, such as in :doc:`../features/distributed_compute_node`
|
||||
and :doc:`../features/deploy_cellv2`, a single inventory file can be
|
||||
generated for all stacks allowing images to be cached on all compute nodes
|
||||
with a single playbook run.
|
||||
|
||||
A multi-stack inventory can be created by specifying a comma separated list of stacks::
|
||||
|
||||
[stack@undercloud-0 ~]$ . stackrc
|
||||
(undercloud) [stack@undercloud-0 ~]$ tripleo-ansible-inventory --plan overcloud,site1,site2 --static-yaml-inventory multiinventory.yaml
|
||||
|
||||
The playbook can then be run once as in :ref:`cache_all_computes` to pre-cache on all compute nodes.
|
||||
|
||||
.. _scp_distribution:
|
||||
|
||||
Pre-caching on one node and distributing to remaining nodes
|
||||
-----------------------------------------------------------
|
||||
|
||||
In the case of a :doc:`../features/distributed_compute_node` it may be desirable to transfer an image to a single compute node at a remote site and then redistribute it from that node to the remaining compute nodes.
|
||||
The SSH/SCP configuration that exists between the compute nodes to support cold migration/resize is reused for this purpose.
|
||||
|
||||
.. warning:: SSH/SCP is inefficient over high latency networks. The method should only be used when the compute nodes targetted by the playbook are all within the same site. To ensure this is the case set tripleo_nova_image_cache_plan to the stack name of the site. Multiple runs of ansible-playbook are then required, targetting a different site each time.
|
||||
|
||||
To enable this simply set `tripleo_nova_image_cache_use_proxy: true` in the arguments file.
|
||||
The image is distributed from the first compute node by default. To use a specific compute node also set `tripleo_nova_image_cache_proxy_hostname`.
|
||||
|
||||
For example::
|
||||
|
||||
(central) [stack@undercloud-0 ~]$ cat <<EOF > dcn1_nova_cache_args.yml
|
||||
tripleo_nova_image_cache_use_proxy: true
|
||||
tripleo_nova_image_cache_proxy_hostname: dcn1-compute-1
|
||||
tripleo_nova_image_cache_images:
|
||||
- id: 07bc2424-753b-4f65-9da5-5a99d8383fe6
|
||||
tripleo_nova_image_cache_plan: dcn1
|
||||
EOF
|
||||
|
||||
(central) [stack@undercloud-0 ~]$ ansible-playbook -i multiinventory.yaml --extra-vars "@dcn1_nova_cache_args.yml" /usr/share/ansible/tripleo-playbooks/tripleo_nova_image_cache.yml
|
||||
|
||||
PLAY [TripleO Nova image cache management] ***************************************************************************************************************************************************************************************************
|
||||
|
||||
TASK [tripleo-nova-image-cache : Show proxy host] ********************************************************************************************************************************************************************************************
|
||||
ok: [dcn-compute-0] => {
|
||||
"msg": "Proxy host is dcn-compute-1"
|
||||
}
|
||||
|
||||
TASK [tripleo-nova-image-cache : Cache image 07bc2424-753b-4f65-9da5-5a99d8383fe6] ***********************************************************************************************************************************************************
|
||||
skipping: [dcn1-compute-0]
|
||||
changed: [dcn1-compute-1]
|
||||
|
||||
TASK [tripleo-nova-image-cache : Cache image (via proxy) 07bc2424-753b-4f65-9da5-5a99d8383fe6] ***********************************************************************************************************************************************
|
||||
skipping: [dcn1-compute-1]
|
||||
changed: [dcn1-compute-0]
|
||||
|
||||
(central) [stack@undercloud-0 ~]$ cat <<EOF > dcn2_nova_cache_args.yml
|
||||
tripleo_nova_image_cache_use_proxy: true
|
||||
tripleo_nova_image_cache_images:
|
||||
- id: 07bc2424-753b-4f65-9da5-5a99d8383fe6
|
||||
tripleo_nova_image_cache_plan: dcn2
|
||||
EOF
|
||||
|
||||
(central) [stack@undercloud-0 ~]$ ansible-playbook -i multiinventory.yaml --extra-vars "@dcn2_nova_cache_args.yml" /usr/share/ansible/tripleo-playbooks/tripleo_nova_image_cache.yml
|
||||
|
||||
PLAY [TripleO Nova image cache management] ***************************************************************************************************************************************************************************************************
|
||||
...
|
||||
...
|
||||
|
||||
.. warning:: The ansible `forks` config option (default=5) will affect the number of concurrent SCP transfers. Consider the load on the proxy compute node if adjusting this.
|
||||
Loading…
Reference in New Issue