Give the OVN DBS service a separate Vip

This change (with its dependent reviews) creates a separate VIP for the OVN DBS
service. A more detailed explanation can be found in https://bugs.launchpad.net/tripleo/+bug/1841811.
The short explanation is that the OVN DBS HA service puts some additional constraints on the VIP it
uses and that is problematic when that VIP is used by other services (e.g. a change in OVN DBS master
will move the VIP and will also reset all mysql connections. It also prevents us splitting OVN DBS from
where haproxy runs).

Tested as follows:
A) Deployed a mster environment with this review and all its dependencies and correctly obtained
an OVN DBS service with its own Vip and the OVN services
(controller/metadata) pointing to this separate Vip

B) Deployed a master environment as is and then applied this review +
dependencies and observed that a redeploy correctly created a new VIP,
reconfigured the services to point to the new VIP and that the old
obsolete constraints created around the per-network VIP were removed

Closes-Bug: #1841811

Depends-On: Ic62b0fbc0fee40638811a5cd77a5dc5a4d82acf5
Change-Id: I620e37117c26b5b51bf9e1eda91daeb00fdf0f43
changes/47/669847/24
Michele Baldessari 3 years ago
parent 41dcc097ba
commit 176b30649b
  1. 3
      deployed-server/deployed-server-environment-output.yaml
  2. 2
      deployment/ovn/ovn-dbs-pacemaker-puppet.yaml
  3. 2
      environments/external-loadbalancer-vip-v6-all.yaml
  4. 2
      environments/external-loadbalancer-vip-v6.yaml
  5. 2
      environments/external-loadbalancer-vip.yaml
  6. 2
      environments/fixed-ip-vips-v6.yaml
  7. 2
      environments/fixed-ip-vips.yaml
  8. 1
      environments/network-isolation-no-tunneling.j2.yaml
  9. 1
      environments/network-isolation-v6-all.j2.yaml
  10. 1
      environments/network-isolation-v6.j2.yaml
  11. 1
      environments/network-isolation.j2.yaml
  12. 1
      environments/standalone.yaml
  13. 1
      environments/standalone/standalone-tripleo.yaml
  14. 1
      environments/undercloud.yaml
  15. 1
      network/service_net_map.j2.yaml
  16. 1
      overcloud-resource-registry-puppet.j2.yaml
  17. 36
      overcloud.j2.yaml
  18. 9
      releasenotes/notes/ovn-separate-vip-ed28ffa3ff77f865.yaml
  19. 1
      sample-env-generator/standalone.yaml

@ -30,6 +30,9 @@ resources:
- redis_virtual_ip:
fixed_ips:
- ip_address: {get_param: [VipMap, redis]}
- ovn_dbs_virtual_ip:
fixed_ips:
- ip_address: {get_param: [VipMap, ovn_dbs]}
DeployedServerEnvironment:
type: OS::Heat::Value

@ -155,7 +155,7 @@ outputs:
list_concat:
- - '/container_puppet_apply.sh'
- '3'
- 'file,file_line,concat,augeas,pacemaker::resource::bundle,pacemaker::property,pacemaker::resource::ip,pacemaker::resource::ocf,pacemaker::constraint::order,pacemaker::constraint::colocation'
- 'file,file_line,concat,augeas,pacemaker::resource::bundle,pacemaker::property,pacemaker::resource::ip,pacemaker::resource::ocf,pacemaker::constraint::order,pacemaker::constraint::colocation,ovn_dbs_remove_old_cruft'
- 'include ::tripleo::profile::base::pacemaker;include ::tripleo::profile::pacemaker::ovn_dbs_bundle'
- if:
- puppet_debug_enabled

@ -7,6 +7,7 @@ resource_registry:
OS::TripleO::Network::Ports::StorageVipPort: ../network/ports/storage_v6.yaml
OS::TripleO::Network::Ports::StorageMgmtVipPort: ../network/ports/storage_mgmt_v6.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Controller::Ports::ExternalPort: ../network/ports/external_from_pool_v6.yaml
OS::TripleO::Controller::Ports::InternalApiPort: ../network/ports/internal_api_from_pool_v6.yaml
OS::TripleO::Controller::Ports::StoragePort: ../network/ports/storage_from_pool_v6.yaml
@ -26,6 +27,7 @@ parameter_defaults:
StorageVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:3000:0000:0000:0000:0005'}]
StorageMgmtVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:4000:0000:0000:0000:0005'}]
RedisVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:2000:0000:0000:0000:0006'}]
OVNDBsVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:2000:0000:0000:0000:0007'}]
ControllerIPs:
external:
- 2001:db8:fd00:1000:0000:0000:0000:0007

@ -4,6 +4,7 @@ resource_registry:
OS::TripleO::Network::Ports::StorageVipPort: ../network/ports/storage_v6.yaml
OS::TripleO::Network::Ports::StorageMgmtVipPort: ../network/ports/storage_mgmt_v6.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Controller::Ports::ExternalPort: ../network/ports/external_from_pool_v6.yaml
OS::TripleO::Controller::Ports::InternalApiPort: ../network/ports/internal_api_from_pool_v6.yaml
OS::TripleO::Controller::Ports::StoragePort: ../network/ports/storage_from_pool_v6.yaml
@ -24,6 +25,7 @@ parameter_defaults:
StorageVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:3000:0000:0000:0000:0005'}]
StorageMgmtVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:4000:0000:0000:0000:0005'}]
RedisVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:2000:0000:0000:0000:0006'}]
OVNDBsVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:2000:0000:0000:0000:0007'}]
ControllerIPs:
external:
- 2001:db8:fd00:1000:0000:0000:0000:0007

@ -4,6 +4,7 @@ resource_registry:
OS::TripleO::Network::Ports::StorageVipPort: ../network/ports/storage.yaml
OS::TripleO::Network::Ports::StorageMgmtVipPort: ../network/ports/storage_mgmt.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip.yaml
OS::TripleO::Controller::Ports::ExternalPort: ../network/ports/external_from_pool.yaml
OS::TripleO::Controller::Ports::InternalApiPort: ../network/ports/internal_api_from_pool.yaml
OS::TripleO::Controller::Ports::StoragePort: ../network/ports/storage_from_pool.yaml
@ -23,6 +24,7 @@ parameter_defaults:
StorageVirtualFixedIPs: [{'ip_address':'172.16.1.251'}]
StorageMgmtVirtualFixedIPs: [{'ip_address':'172.16.3.251'}]
RedisVirtualFixedIPs: [{'ip_address':'172.16.2.252'}]
OVNDBsVirtualFixedIPs: [{'ip_address':'172.16.2.253'}]
ControllerIPs:
external:
- 10.0.0.253

@ -8,6 +8,7 @@ resource_registry:
OS::TripleO::Network::Ports::StorageVipPort: ../network/ports/storage_v6.yaml
OS::TripleO::Network::Ports::StorageMgmtVipPort: ../network/ports/storage_mgmt_v6.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip_v6.yaml
parameter_defaults:
# Set the IP addresses of the VIPs here.
@ -19,3 +20,4 @@ parameter_defaults:
StorageVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:3000:0000:0000:0000:000'}]
StorageMgmtVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:4000:0000:0000:0000:0005'}]
RedisVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:2000:0000:0000:0000:0006'}]
OVNDBsVirtualFixedIPs: [{'ip_address':'fd00:fd00:fd00:2000:0000:0000:0000:0007'}]

@ -8,6 +8,7 @@ resource_registry:
OS::TripleO::Network::Ports::StorageVipPort: ../network/ports/storage.yaml
OS::TripleO::Network::Ports::StorageMgmtVipPort: ../network/ports/storage_mgmt.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip.yaml
parameter_defaults:
# Set the IP addresses of the VIPs here.
@ -19,3 +20,4 @@ parameter_defaults:
StorageVirtualFixedIPs: [{'ip_address':'172.16.1.240'}]
StorageMgmtVirtualFixedIPs: [{'ip_address':'172.16.3.240'}]
RedisVirtualFixedIPs: [{'ip_address':'172.16.2.241'}]
OVNDBsVirtualFixedIPs: [{'ip_address':'172.16.2.242'}]

@ -20,6 +20,7 @@ resource_registry:
OS::TripleO::Network::Ports::{{network.name}}VipPort: ../network/ports/{{network.name_lower|default(network.name.lower())}}.yaml
{%- endfor %}
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip.yaml
# Port assignments for each role are determined by the role definition.
{%- for role in roles %}

@ -34,6 +34,7 @@ resource_registry:
{%- endfor %}
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip_v6.yaml
# Port assignments by role, edit role definition to assign networks to roles.
{%- for role in roles %}

@ -27,6 +27,7 @@ resource_registry:
{%- endfor %}
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip_v6.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip_v6.yaml
# Port assignments by role, edit role definition to assign networks to roles.
{%- for role in roles %}

@ -16,6 +16,7 @@ resource_registry:
{%- endif %}
{%- endfor %}
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/vip.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/vip.yaml
# Port assignments by role, edit role definition to assign networks to roles.
{%- for role in roles %}

@ -3,6 +3,7 @@
resource_registry:
OS::TripleO::Services::Tmpwatch: ../deployment/logrotate/tmpwatch-install.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/noop.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/noop.yaml
OS::TripleO::Network::Ports::ControlPlaneVipPort: ../deployed-server/deployed-neutron-port.yaml
OS::TripleO::Standalone::Net::SoftwareConfig: ../net-config-standalone.yaml
OS::TripleO::NodeExtraConfigPost: ../extraconfig/post_deploy/standalone_post.yaml

@ -51,6 +51,7 @@ parameter_defaults:
resource_registry:
OS::TripleO::Network::Ports::ControlPlaneVipPort: ../../deployed-server/deployed-neutron-port.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../../network/ports/noop.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../../network/ports/noop.yaml
OS::TripleO::NodeExtraConfigPost: ../../extraconfig/post_deploy/standalone_post.yaml
OS::TripleO::Services::AodhApi: OS::Heat::None
OS::TripleO::Services::AodhEvaluator: OS::Heat::None

@ -5,6 +5,7 @@ parameter_merge_strategies:
resource_registry:
OS::TripleO::Services::Tmpwatch: ../deployment/logrotate/tmpwatch-install.yaml
OS::TripleO::Network::Ports::RedisVipPort: ../network/ports/noop.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../network/ports/noop.yaml
OS::TripleO::Network::Ports::ControlPlaneVipPort: ../deployed-server/deployed-neutron-port.yaml
OS::TripleO::Undercloud::Net::SoftwareConfig: ../net-config-undercloud.yaml
OS::TripleO::NodeExtraConfigPost: ../extraconfig/post_deploy/undercloud_post.yaml

@ -123,6 +123,7 @@ parameters:
{{network.name}}: {{network.name_lower}}_subnet
{%- endfor %}
redis: {{ _service_nets.get('internal_api', 'internal_api') }}_subnet
ovn_dbs: {{ _service_nets.get('internal_api', 'internal_api') }}_subnet
type: json
# We define mappings to work around names that break when doing the

@ -85,6 +85,7 @@ resource_registry:
{%- endfor %}
OS::TripleO::Network::Ports::RedisVipPort: network/ports/ctlplane_vip.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: network/ports/ctlplane_vip.yaml
OS::TripleO::Network::Ports::ControlPlaneVipPort: OS::Neutron::Port
# Service to network Mappings

@ -123,6 +123,12 @@ parameters:
Control the IP allocation for the virtual IP used by Redis. E.g.
[{'ip_address':'1.2.3.4'}]
type: json
OVNDBsVirtualFixedIPs:
default: []
description: >
Control the IP allocation for the virtual IP used by OVN DBs. E.g.
[{'ip_address':'1.2.3.4'}]
type: json
CloudDomain:
default: 'localdomain'
type: string
@ -332,6 +338,11 @@ conditions:
equals:
- get_param: RedisVirtualFixedIPs
- []
ovn_dbs_virtual_fixed_ip_set:
not:
equals:
- get_param: OVNDBsVirtualFixedIPs
- []
set_default_mysql_cell_internal:
or:
- equals:
@ -923,6 +934,26 @@ resources:
- {get_param: RedisVirtualFixedIPs}
- [{subnet: {get_attr: [ServiceNetMap, vip_subnet_map, redis]}}]
OVNDBsVirtualIP:
depends_on: [Networks, ServiceNetMap]
type: OS::TripleO::Network::Ports::OVNDBsVipPort
properties:
ControlPlaneIP: {get_attr: [ControlVirtualIP, fixed_ips, 0, ip_address]}
ControlPlaneSubnetCidr:
if:
- ctlplane_subnet_cidr_set
- {get_param: ControlPlaneSubnetCidr}
- {str_split: ['/', {get_attr: [ControlVirtualIP, subnets, 0, cidr]}, 1]}
ControlPlaneNetwork: {get_param: NeutronControlPlaneID}
PortName: ovn_dbs_virtual_ip
NetworkName: {get_attr: [ServiceNetMap, service_net_map, OvnDbsNetwork]}
ServiceName: ovn_dbs
FixedIPs:
if:
- ovn_dbs_virtual_fixed_ip_set
- {get_param: OVNDBsVirtualFixedIPs}
- [{subnet: {get_attr: [ServiceNetMap, vip_subnet_map, ovn_dbs]}}]
{%- for network in networks if network.vip|default(false) and network.enabled|default(true) %}
{%- if network.name == 'External' %}
# The public VIP is on the External net, falls back to ctlplane
@ -1077,9 +1108,9 @@ resources:
map_merge:
- {get_attr: [VipMap, net_ip_map]}
- redis: {get_attr: [RedisVirtualIP, ip_address]}
- ovn_dbs: {get_attr: [OVNDBsVirtualIP, ip_address]}
CloudNames: {get_attr: [CloudNames, value]}
outputs:
ManagedEndpoints:
description: Asserts that the keystone endpoints have been provisioned.
@ -1159,11 +1190,12 @@ outputs:
{{role.name}}: {{role.tags|default([])}}
{%- endfor %}
VipMap:
description: Mapping of each network to VIP addresses. Also includes the Redis VIP.
description: Mapping of each network to VIP addresses. Also includes the Redis and OVN DBs VIPs.
value:
map_merge:
- {get_attr: [VipMap, net_ip_map]}
- redis: {get_attr: [RedisVirtualIP, ip_address]}
- ovn_dbs: {get_attr: [OVNDBsVirtualIP, ip_address]}
ServerIdData:
description: Mapping of each role to a list of nova server IDs and the bootstrap ID
value: {get_attr: [ServerIdMap, value]}

@ -0,0 +1,9 @@
---
fixes:
- |
This change (with its dependent reviews) creates a separate VIP for the OVN DBS
service. A more detailed explanation can be found in https://bugs.launchpad.net/tripleo/+bug/1841811.
The short explanation is that the OVN DBS HA service puts some additional constraints on the VIP it
uses and that is problematic when that VIP is used by other services (e.g. a change in OVN DBS master
will move the VIP and will also reset all mysql connections. It also prevents us splitting OVN DBS from
where haproxy runs).

@ -51,6 +51,7 @@ environments:
resource_registry:
# this network config is assumed by the tripleo deploy command
OS::TripleO::Network::Ports::RedisVipPort: ../../network/ports/noop.yaml
OS::TripleO::Network::Ports::OVNDBsVipPort: ../../network/ports/noop.yaml
OS::TripleO::Network::Ports::ControlPlaneVipPort: ../../deployed-server/deployed-neutron-port.yaml
OS::TripleO::Standalone::Net::SoftwareConfig: ../../net-config-standalone.yaml
OS::TripleO::NodeExtraConfigPost: ../../extraconfig/post_deploy/standalone_post.yaml

Loading…
Cancel
Save