Avoid unconditional fact gathering

One way to improve the performance of Ansible is through fact caching.
Rather than gather facts in every play, we can configure Ansible to
cache them in a persistent store. An example Ansible configuration for
doing this is as follows:

[defaults]
gathering = smart
fact_caching = jsonfile
fact_caching_connection = ./facts
fact_caching_timeout = 86400

This does not affect Kolla Ansible however, since we use the setup
module which unconditionally gathers facts regardless of the state of
the cache. This gets worse with large inventories limited to a small
batch of hosts via --limit or serial, since the limited hosts must
gather facts for all others.

One way to detect whether facts exist for a host is via the
'module_setup' variable, which exists only when facts exist. This change
uses the 'module_setup' fact to determine whether facts need to be
gathered for hosts outside of the batch. For hosts in the batch, we
switch from using the setup module to gather_facts on the play, which
can use the 'smart' gathering logic.

Change-Id: I04841fb62b2e1d9e97ce4b75ce3a7349b9c74036
Partially-Implements: blueprint performance-improvements

ansible/gather-facts.yml

@@ -1,17 +1,12 @@
 ---
 # NOTE(awiddersheim): Gather facts for all hosts as a
 # first step since several plays below require them when
-# building their configurations. The below 'gather_facts'
+# building their configurations.
-# set to 'false' is a bit confusing but this is to avoid
-# Ansible gathering facts twice.
 - name: Gather facts for all hosts
   hosts: all
   serial: '{{ kolla_serial|default("0") }}'
-  gather_facts: false
+  gather_facts: true
   tasks:
-    - name: Gather facts
-      setup:
-
     - name: Group hosts to determine when using --limit
       group_by:
         key: "all_using_limit_{{ (ansible_play_batch | length) != (groups['all'] | length) }}"
@@ -40,5 +35,6 @@
       delegate_to: "{{ item }}"
       with_items: "{{ delegate_hosts }}"
       # We gathered facts for all hosts in the batch during the first play.
-      when: item not in ansible_play_batch
+      when:
+        - not hostvars[item].module_setup | default(false)
   tags: always

releasenotes/notes/avoid-unconditional-fact-gathering-94760984b2de0796.yaml

@@ -0,0 +1,8 @@
+---
+upgrade:
+  - |
+    Avoids unnecessary fact gathering using the ``setup`` module. This should
+    improve the performance of environments using fact caching and the Ansible
+    ``smart`` fact gathering policy. See `blueprint
+    <https://blueprints.launchpad.net/kolla-ansible/+spec/performance-improvements>`__
+    for details.