Support reducing scope of delegated fact gathering

When Kolla Ansible is executed with a '--limit' argument, the scope of
an operation is limited to the hosts in the limit. For example:

   kolla-ansible deploy --limit control

Due to the nature of configuring clustered software services, there are
cases where we need to know information about other hosts. Most often
this is related to their hostname or network addresses. To make this
work, Kolla Ansible gathers facts for hosts outside of the limit using
delegated fact gathering [1].

By default, Kolla Ansible gathers facts for all hosts. Because delegated
facts are gathered serially in batches by the active hosts, this can
take a long time when there are not many hosts in the limit.

This change makes it possible to reduce the set of hosts eligible for
delegated fact gathering by setting 'kolla_ansible_delegate_facts_hosts'
to a list of hosts.

[1] https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_delegation.html#delegating-facts

Depends-On: https://review.opendev.org/c/openstack/kolla-ansible/+/899615

Change-Id: Id0bda00b81b5bc6a9a870e231335c33d9828e95f

ansible/gather-facts.yml

@@ -44,7 +44,7 @@
     batch_size: "{{ ansible_play_batch | length }}"
     # Use a python list slice to divide the group up.
     # Syntax: [<start index>:<end index>:<step size>]
-    delegate_hosts: "{{ groups['all'][batch_index | int::batch_size | int] }}"
+    delegate_hosts: "{{ kolla_ansible_delegate_facts_hosts[batch_index | int::batch_size | int] }}"
   tasks:
     - name: Gather facts
       setup:

ansible/group_vars/all.yml

@@ -36,6 +36,12 @@ kolla_ansible_setup_filter: "{{ omit }}"
 # By default, we do not provide a gather subset.
 kolla_ansible_setup_gather_subset: "{{ omit }}"
 
+# This variable determines which hosts require facts when using --limit. Facts
+# will be gathered using delegation for hosts in this list that are not
+# included in the limit.
+# By default, this list includes all hosts.
+kolla_ansible_delegate_facts_hosts: "{{ groups['all'] }}"
+
 ###################
 # Kolla options
 ###################

doc/source/user/ansible-tuning.rst

@@ -160,3 +160,44 @@ A max fail percentage may be set for specific services using
 
    kolla_max_fail_percentage: 50
    nova_max_fail_percentage: 25
+
+Delegated fact gathering
+------------------------
+
+When Kolla Ansible is executed with a ``--limit`` argument, the scope of an
+operation is limited to the hosts in the limit. For example:
+
+.. code-block:: console
+
+   kolla-ansible deploy --limit control
+
+Due to the nature of configuring clustered software services, there are cases
+where we need to know information about other hosts. Most often this is related
+to their hostname or network addresses. To make this work, Kolla Ansible
+gathers facts for hosts outside of the limit using `delegated fact gathering
+<https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_delegation.html#delegating-facts>`__.
+
+By default, Kolla Ansible gathers facts for all hosts. Because delegated facts
+are gathered serially in batches by the active hosts, this can take a long time
+when there are not many hosts in the limit. If you know that facts are not
+required for all hosts, it is possible to reduce the set of hosts eligible for
+delegated fact gathering by setting ``kolla_ansible_delegate_facts_hosts`` to a
+list of hosts. This may be done permanently in ``globals.yml`` or temporarily
+for the duration of a command using the ``-e`` argument.
+
+The exact requirements will depend upon configuration and inventory, but here
+are some rules of thumb:
+
+* Facts are typically required for all controllers, regardless of which hosts
+  are in the limit. This is due to references to RabbitMQ and Memcache
+  connection strings etc.
+* Prometheus server requires facts for all other hosts to generate scrape
+  configs for node exporter, cAdvisor, etc. Specifically it uses the IP address
+  of the API interface. This may be avoided by hard-coding
+  ``prometheus_target_address`` in the inventory for each host.
+* Configuration of ``/etc/hosts`` during the ``bootstrap-servers`` command
+  requires facts for all other hosts. Specifically it uses the IP address of
+  the API interface, and the ``hostname`` and ``nodename`` facts.
+* Noting the above exceptions, compute nodes are fairly independent. Other
+  hosts do not need to know their facts, and they do not need to know other
+  hosts' facts.

etc/kolla/globals.yml

@@ -5,6 +5,9 @@
 # commented parameters are shown here, To override the default value uncomment
 # the parameter and change its value.
 
+# Dummy variable to allow Ansible to accept this file.
+workaround_ansible_issue_8743: yes
+
 ###################
 # Ansible options
 ###################
@@ -21,15 +24,18 @@
 # By default, we do not provide a gather subset.
 #kolla_ansible_setup_gather_subset: "{{ omit }}"
 
-# Dummy variable to allow Ansible to accept this file.
-workaround_ansible_issue_8743: yes
-
 # This variable may be used to set the maximum failure percentage for all
 # plays. More fine-grained control is possible via per-service variables, e.g.
 # nova_max_fail_percentage. The default behaviour is to set a max fail
 # percentage of 100, which is equivalent to not setting it.
 #kolla_max_fail_percentage:
 
+# This variable determines which hosts require facts when using --limit. Facts
+# will be gathered using delegation for hosts in this list that are not
+# included in the limit.
+# By default, this list includes all hosts.
+#kolla_ansible_delegate_facts_hosts: "{{ groups['all'] }}"
+
 ###############
 # Kolla options
 ###############

releasenotes/notes/delegate-facts-hosts-a3c8bd588c805ffa.yaml

@@ -0,0 +1,6 @@
+---
+features:
+  - |
+    Adds a new variable, ``kolla_ansible_delegate_facts_hosts``, that may be
+    used to control which hosts require facts when using ``--limit``. Its
+    default value is ``groups['all']``.