Use controller-0 ip for calico-node can-reach dest

Because cluster floating IP host can change, it should not be used for
calico node address, and doing so has been observed to cause an error
when calico-node is rebooted on the active controller and then a swact
is executed, causing BGP peering to be lost.

This commit switches to using controller-0 cluster host address for
route selection, in order to ensure a consistent route selection is
used.

Closes-Bug: 1885582
Change-Id: I56c5ddf657eb557b83ce0fd3ce7beb71011d6266
Signed-off-by: Joseph Richard <joseph.richard@windriver.com>

playbookconfig/src/playbooks/roles/bootstrap/bringup-essential-services/templates/k8s-v1.18.1/calico-cni.yaml.j2

@@ -665,7 +665,7 @@ spec:
             - name: IP
               value: "autodetect"
             - name: IP_AUTODETECTION_METHOD
-              value: "can-reach={{ cluster_floating_address }}"
+              value: "can-reach={{ controller_0_cluster_host }}"
             # Enable IPIP
             - name: CALICO_IPV4POOL_IPIP
               value: "Always"
@@ -685,7 +685,7 @@ spec:
             - name: IP6
               value: "autodetect"
             - name: IP6_AUTODETECTION_METHOD
-              value: "can-reach={{ cluster_floating_address }}"
+              value: "can-reach={{ controller_0_cluster_host }}"
             # The default IPv6 pool to create on startup if none exists. Pod IPs will be
             # chosen from this range. Changing this value after installation will have
             # no effect. This should fall within `--cluster-cidr`.

playbookconfig/src/playbooks/upgrade-k8s-networking.yml

@@ -30,6 +30,7 @@
         cluster_network_ipv4: "{{ cluster_pod_subnet | ipv4 }}"
         cluster_network_ipv6: "{{ cluster_pod_subnet | ipv6 }}"
         cluster_floating_address: "{{ cluster_host_floating_address }}"
+        controller_0_cluster_host: "{{ cluster_host_node_0_address }}"
 
     - name: Upgrade k8s networking images
       import_role: