Implement Ansible fact cache for Mistral executor

Most of the playbooks executed by Mistral gather facts which, by default,
are stored in memory until the conclusion of the playbook execution. This
means that for every new playbook run the facts for each host are gathered
again. Fact gathering can take a long time, especially when the target host
has a lot of devices, which is often the case for compute hosts (many
network devices and many storage devices) and for controller hosts (many
network devices).

To optimise fact gathering, we set the default behaviour for Mistral's
Ansible execution to store gathered facts using the jsonfile format in a
temporary location, and we tell it to use smart gathering so that it will
only gather new facts when a new playbook executes.

The validations are run using the 'validations' user, it cannot share the
same fact cache because the /var/tmp/ansible_fact_cache folder will be owned
by the 'mistral' user, and is therefore not writable by the 'validations'
user. We therefore set the validations to use a different path.

This should cut down the deployment time a bit, which saves a little time in
CI and saves a lot of time in production environments.

Related: rhbz#1749406
Change-Id: Ie7adf41cef0fafde646f840b2bf3e01ba42a17d5

scripts/run-validation

@@ -67,6 +67,14 @@ fi
 export ANSIBLE_HOST_KEY_CHECKING=False
 # Disable retry files until we find a good use and location for them
 export ANSIBLE_RETRY_FILES_ENABLED=False
+# Use a fact cache to improve performance
+# The defaults from tripleo_common/actions/ansible.py cannot be used
+# beause they will clash with the 'mistral' user owning the fact cache
+# path.
+export ANSIBLE_GATHERING=smart
+export ANSIBLE_CACHE_PLUGIN=jsonfile
+export ANSIBLE_CACHE_PLUGIN_CONNECTION=~/ansible_fact_cache
+export ANSIBLE_GATHER_TIMEOUT=7200 # 7200s = 2h
 # ANSIBLE_SUDO_FLAGS is deprecated in favor of become and will be removed in
 # version 2.8. We now have to use ANSIBLE_BECOME_FLAGS to pass Flags to the
 # privilege escalation sudo executable.

tripleo_common/actions/ansible.py

@@ -106,6 +106,15 @@ def write_default_ansible_cfg(work_dir,
     config.set('defaults', 'timeout', '30')
     config.set('defaults', 'gather_timeout', '30')
 
+    # Setup fact cache to improve playbook execution speed
+    config.set('defaults', 'gathering', 'smart')
+    config.set('defaults', 'fact_caching', 'jsonfile')
+    config.set('defaults', 'fact_caching_connection',
+               '/var/tmp/ansible_fact_cache')
+
+    # Expire facts in the fact cache after 7200s (2h)
+    config.set('defaults', 'fact_caching_timeout', '7200')
+
     # mistral user has no home dir set, so no place to save a known hosts file
     config.set('ssh_connection', 'ssh_args',
                '-o UserKnownHostsFile=/dev/null '