Add ping test for all networks gateway IPs

Add ping test for gateway IPs on all networks, to ensure
all gateways are reachable.

The releated Bugzilla reports an issue where some network
fabrics fail when using the current node ping test, which
pings the first node in each role. The fabric simply does
not forward traffic before the gateway has been pinged.

One can argue that the fabric in question is broken. However,
with the current implementation the first node in each role
actually ping tests only against it's own address? So adding
the test to ping the gateway addresses improves the validation
in general.

Make PingTestGatewayIPsMap a map of flatten lists

PingTestGatewayIPsMap elements may contain list of lists, causing
failures on roles that iterate over them. See [1] and #1950528 for
more info.

[1] https://review.opendev.org/c/openstack/tripleo-ansible/+/817500

Add attr of networks and subnets to Networks resource

In the Networks resource tempaltes add the full resource
attributes to net_attributes_map. This is a partial cherry-pick of
commit 5b3878580a.

Also the parameter {{role.name}}ControlPlaneSubnet is added in
overcloud.yaml.j2, the parameter was previously only used in
puppet/role.role.j2.yaml. This is a partial cherry-pick of
commit 7b8c6b07da.

Conflicts:
  common/deploy-steps.j2

Closes-bug: #1950528
Related RHBZ#1875962
Depends-On: I93cded61ffb862e99fd8043dbf0def3d16079692
Depends-On: https://review.opendev.org/825676
Change-Id: I3309f2a0e39ad115930ecd5c0e895816565819e9
(cherry picked from commit 5d830980ec)
(cherry picked from commit efc328c668)

common/deploy-steps.j2

@@ -130,6 +130,10 @@ parameters:
     default: false
     description: Optional validation to ensure FQDN as set by Nova matches the name set in /etc/hosts.
     type: boolean
+  PingTestGatewayIPsMap:
+    default: {}
+    description: A map of role name to list of gateway IP addresses for each network, used to ping test each gateway.
+    type: json
   PingTestIpsMap:
     default: ''
     description: A map of role name to a space separated list of IP addresses used to ping test each available network interface.
@@ -350,6 +354,7 @@ outputs:
         validate_controllers_icmp: {get_param: ValidateControllersIcmp}
         validate_gateways_icmp: {get_param: ValidateGatewaysIcmp}
         validate_fqdn: {get_param: ValidateFqdn}
+        ping_test_gateway_ips: {get_param: PingTestGatewayIPsMap}
         ping_test_ips: {get_param: PingTestIpsMap}
         stack_action: {get_param: StackAction}
         deploy_artifact_urls: {list_join: [' ', {get_param: DeployArtifactURLs}]}
@@ -590,6 +595,7 @@ outputs:
                     tripleo_nodes_validation_validate_gateway_icmp: "{{ validate_gateways_icmp }}"
                     tripleo_nodes_validation_validate_fqdn: "{{ validate_fqdn }}"
                     tripleo_nodes_validation_ping_test_ips: "{{ ping_test_ips.get(tripleo_role_name).split(' ') | list | unique }}"
+                    tripleo_nodes_validation_ping_test_gateway_ips: "{{ ping_test_gateway_ips.get(tripleo_role_name) }}"
                 - name: ArtifactsConfig
                   script: deploy-artifacts.sh
                   environment:

network/network.j2

@@ -274,4 +274,14 @@ outputs:
         - - {get_param: {{network.name}}NetCidr}
 {%- for subnet in network.subnets|default({}) if network.subnets[subnet].enabled|default(true) %}
           - {get_param: {{network.name}}SubnetCidr_{{subnet}}}
+{%- endfor %}
+  network_attrs:
+    description: {{network.name}} network's attributes
+    value: {get_attr: [{{network.name}}Network]}
+  subnet_attrs:
+    description: Map of attributes for subnets in the {{network.name}} network
+    value:
+      {{network.name_lower}}_subnet: {get_attr: [{{network.name}}Subnet]}
+{%- for subnet in network.subnets|default({}) if network.subnets[subnet].enabled|default(true) %}
+      {{subnet}}: {get_attr: [{{network.name}}Subnet_{{subnet}}]}
 {%- endfor %}

network/networks.j2.yaml

@@ -36,5 +36,13 @@ outputs:
     value:
       {%- for network in networks if network.enabled|default(true) %}
       {%- set network_name = network.compat_name|default(network.name) %}
-        {{network.name_lower}}: {get_attr: [{{network_name}}Network, ip_version]}
+      {{network.name_lower}}: {get_attr: [{{network_name}}Network, ip_version]}
+      {%- endfor %}
+  net_attributes_map:
+    value:
+      {%- for network in networks if network.enabled|default(true) %}
+      {%- set network_name = network.compat_name|default(network.name) %}
+      {{network.name_lower}}:
+        network: {get_attr: [{{network_name}}Network, network_attrs]}
+        subnets: {get_attr: [{{network_name}}Network, subnet_attrs]}
       {%- endfor %}

overcloud.j2.yaml

@@ -276,6 +276,12 @@ parameters:
     description: Optional extra Ansible group vars
     default: {}
 
+  {{role.name}}ControlPlaneSubnet:
+    default: ctlplane-subnet
+    description: |
+      Name of the subnet on ctlplane network for this role.
+    type: string
+
   {{role.name}}ServiceNetMap:
     default: {}
     description: |
@@ -1108,6 +1114,24 @@ resources:
       {{role.name}}Count: {get_param: {{role.name}}Count}
 {%- endfor %}
       ServiceNetMapLower: {get_attr: [ServiceNetMap, service_net_map_lower]}
+      PingTestGatewayIPsMap:
+{%- for role in roles %}
+        {{role.name}}:
+          yaql:
+            expression: list($.data.where($ != null)).flatten()
+            data:
+              - {get_attr: [Networks, net_attributes_map, ctlplane, subnets, {get_param: {{role.name}}ControlPlaneSubnet}, gateway_ip]}
+    {%- for network in networks %}
+      {%- if network.enabled|default(true) and network.name in role.networks|default([]) %}
+        {%- if role.networks is mapping %}
+          {%- set _role_net_subnet = role.networks[network.name]['subnet'] %}
+        {%- else %}
+          {%- set _role_net_subnet = network.name_lower + '_subnet' %}
+        {%- endif %}
+              - {get_attr: [Networks, net_attributes_map, {{network.name_lower}}, subnets, {{ _role_net_subnet }}, gateway_ip]}
+      {%- endif %}
+    {%- endfor %}
+{%- endfor %}
       PingTestIpsMap:
 {%- for role in roles %}
         {{role.name}}: