191 lines
7.8 KiB
ObjectPascal
Raw Normal View History

# Copyright 2014 Red Hat, Inc.
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# == Class: tripleo::haproxy::endpoint
#
# Configure a HAProxy listen endpoint
#
# [*internal_ip*]
# The IP in which the proxy endpoint will be listening in the internal
# network.
#
# [*ip_addresses*]
# The ordered list of IPs to be used to contact the balancer member.
#
# [*server_names*]
# The names of the balancer members, which usually should be the hostname.
#
# [*member_options*]
# Options for the balancer member, specified after the server declaration.
# These should go in the member's configuration block.
#
# [*public_virtual_ip*]
# Address in which the proxy endpoint will be listening in the public network.
# If this service is internal only this should be ommitted.
# Defaults to undef.
#
# [*haproxy_listen_bind_param*]
# A list of params to be added to the HAProxy listener bind directive.
# Defaults to undef.
#
# [*public_certificate*]
# Certificate path used to enable TLS for the public proxy endpoint.
# Defaults to undef.
#
# [*use_internal_certificates*]
# Flag that indicates if we'll use an internal certificate for this specific
# service. When set, enables SSL on the internal API endpoints using the file
# that certmonger is tracking; this is derived from the network the service is
# listening on.
# Defaults to false
#
# [*internal_certificates_specs*]
# A hash that should contain the specs that were used to create the
# certificates. As the name indicates, only the internal certificates will be
# fetched from here. And the keys should follow the following pattern
# "haproxy-<network name>". The network name should be as it was defined in
# tripleo-heat-templates.
# Note that this is only taken into account if the $use_internal_certificates
# flag is set.
# Defaults to {}
#
# [*service_network*]
# (optional) Indicates the network that the service is running on. Used for
# fetching the certificate for that specific network.
# Defaults to undef
#
Fix horizon firewall rules in composable roles Atm horizon haproxy firewall rules obfuscate any other rule defined via the tripleo.haproxy.firewall_rules key. Things broke with https://review.opendev.org/#/c/625600/. The reason that was pushed is that in composable roles, when splitting off horizon away from where haproxy runs, we would not have the proper iptables rules on the haproxy role. This was due to the fact that we had the following code: service_config_settings: haproxy: tripleo.horizon.firewall_rules: '127 horizon': dport: - 80 - 443 The above code never worked as explained in 3f8ce6fd96bc4f28a052b4c87a19b4b152734091 and so we fixed it by setting the proper tripleo.haproxy.firewall_rules key. The issue is that rules for haproxy should just never have been set at all via service_config_settings keys in the first place. As demonstrated with this bug, the merging of hiera dictionaries will mess us up and we'll end up overwriting other keys. Haproxy stats access has this: outputs: role_data: description: Role data for the HAproxy role. value: service_name: haproxy monitoring_subscription: {get_param: MonitoringSubscriptionHaproxy} config_settings: map_merge: - tripleo.haproxy.firewall_rules: '107 haproxy stats': dport: 1993 And since hiera will return the horizon settings for tripleo.haproxy.firewall_rules which won't be deep merged with the firewall rules from haproxy stats and so rule '107 haproxy stats' will never be present. Rules for haproxy need to happen in puppet-tripleo/manifests/haproxy*. Normally they do, the exception is horizon which uses a specialized horizon_endpoint.pp manifest which does not trigger these rules. Let's create the firewall rules in haproxy/horizon_endpoint.pp like we do for all other endpoints. Tested and correctly got: [root@controller-0 ~]# iptables -nvL |grep hor 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 80 state NEW /* 100 horizon_haproxy ipv4 */ 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 443 state NEW /* 100 horizon_haproxy_ssl ipv4 */ 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 80,443 state NEW /* 126 horizon ipv4 */ Change-Id: I1325171ef60d7a7e3b57373082fcdb5487be939b Related-Bug: #1829338
2019-05-16 09:12:50 +02:00
# [*manage_firewall*]
# (optional) Enable or disable firewall settings for ports exposed by HAProxy
# (false means disabled, and true means enabled)
# Defaults to hiera('tripleo::firewall::manage_firewall', true)
#
class tripleo::haproxy::horizon_endpoint (
$internal_ip,
$ip_addresses,
$server_names,
$member_options,
$public_virtual_ip,
$haproxy_listen_bind_param = undef,
$public_certificate = undef,
$use_internal_certificates = false,
$internal_certificates_specs = {},
$service_network = undef,
Fix horizon firewall rules in composable roles Atm horizon haproxy firewall rules obfuscate any other rule defined via the tripleo.haproxy.firewall_rules key. Things broke with https://review.opendev.org/#/c/625600/. The reason that was pushed is that in composable roles, when splitting off horizon away from where haproxy runs, we would not have the proper iptables rules on the haproxy role. This was due to the fact that we had the following code: service_config_settings: haproxy: tripleo.horizon.firewall_rules: '127 horizon': dport: - 80 - 443 The above code never worked as explained in 3f8ce6fd96bc4f28a052b4c87a19b4b152734091 and so we fixed it by setting the proper tripleo.haproxy.firewall_rules key. The issue is that rules for haproxy should just never have been set at all via service_config_settings keys in the first place. As demonstrated with this bug, the merging of hiera dictionaries will mess us up and we'll end up overwriting other keys. Haproxy stats access has this: outputs: role_data: description: Role data for the HAproxy role. value: service_name: haproxy monitoring_subscription: {get_param: MonitoringSubscriptionHaproxy} config_settings: map_merge: - tripleo.haproxy.firewall_rules: '107 haproxy stats': dport: 1993 And since hiera will return the horizon settings for tripleo.haproxy.firewall_rules which won't be deep merged with the firewall rules from haproxy stats and so rule '107 haproxy stats' will never be present. Rules for haproxy need to happen in puppet-tripleo/manifests/haproxy*. Normally they do, the exception is horizon which uses a specialized horizon_endpoint.pp manifest which does not trigger these rules. Let's create the firewall rules in haproxy/horizon_endpoint.pp like we do for all other endpoints. Tested and correctly got: [root@controller-0 ~]# iptables -nvL |grep hor 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 80 state NEW /* 100 horizon_haproxy ipv4 */ 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 443 state NEW /* 100 horizon_haproxy_ssl ipv4 */ 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 80,443 state NEW /* 126 horizon ipv4 */ Change-Id: I1325171ef60d7a7e3b57373082fcdb5487be939b Related-Bug: #1829338
2019-05-16 09:12:50 +02:00
$manage_firewall = hiera('tripleo::firewall::manage_firewall', true),
) {
Allow custom per-service listen_options for haproxy There are situations where it would be advantageous to let an operator specify custom per-service options. One such use case seen in the wild is to extend the timeout of the cinder because due to the specific storage backend these cinder operations sometimes take a bit longer. Letting the user tweak the haproxy_default_timeout is likely not what we want as for the case above we only want to tweak a single service. We explored another approach to fix this by adding a bunch of <service>_options class parameters in the tripleo::haproxy class but it made it extremely bloated and confusing, so we opted for this approach which is much less invasive both code-wise and complexity-wise Tested by deploying with: ExtraConfig: tripleo::haproxy::cinder::options: "timeout client": '90m' 'timeout server': '90m' And observing the following cinder haproxy stanza: listen cinder bind 10.0.0.4:8776 transparent bind 172.16.2.9:8776 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } option httpchk option httplog timeout client 90m timeout server 90m server overcloud-controller-0.internalapi.localdomain 172.16.2.7:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.16.2.16:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.16.2.13:8776 check fall 5 inter 2000 rise 2 Closes-Bug: #1755711 Change-Id: Icb7f026190b310d34c47dc059e2fdb22031b0963
2018-03-14 08:31:40 +01:00
# Let users override the options on a per-service basis
$custom_options = hiera('tripleo::haproxy::horizon::options', undef)
Allow custom per-service bind_options for haproxy There are situation when it might be required to use different TLS versions between the services. HAproxy configures TLS version on bind line in the configuration, there is missing customization. At the moment we can only set TLS version globally via ssl_options This code's idea it to configure it per-service. For example, with: parameter_defaults: ExtraConfig: tripleo::haproxy::cinder::internal_bind_options: 'force-tlsv11' tripleo::haproxy::keystone_public::public_bind_options: 'force-tlsv12' tripleo::haproxy::horizon::public_bind_options: 'force-tlsv11' tripleo::haproxy::horizon::internal_bind_options: 'force-tlsv12' We will get something like the following in the haproxy config: listen cinder bind 2620:52:0:13b8:5054:ff:fe3e:1:13776 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem bind fd00:fd00:fd00:2000::17:8776 transparent force-tlsv11 ... listen keystone_public bind 2620:52:0:13b8:5054:ff:fe3e:1:13000 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv12 bind fd00:fd00:fd00:2000::17:5000 transparent ... listen horizon bind 2620:52:0:13b8:5054:ff:fe3e:1:443 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv11 bind 2620:52:0:13b8:5054:ff:fe3e:1:80 transparent force-tlsv11 bind fd00:fd00:fd00:2000::17:443 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv12 bind fd00:fd00:fd00:2000::17:80 transparent force-tlsv12 ... The two {public,internal}_bind_options accept both strings and arrays of strings. Closes-Bug: #1829328 Change-Id: I4b724a515d729c2e8e0da9cb8f081b8325d51a6b
2019-05-18 14:16:47 +02:00
$custom_bind_options_public = delete(any2array(hiera('tripleo::haproxy::horizon::public_bind_options', undef)), undef).flatten()
$custom_bind_options_internal = delete(any2array(hiera('tripleo::haproxy::horizon::internal_bind_options', undef)), undef).flatten()
# service exposed to the public network
if $public_certificate {
if $use_internal_certificates {
if !$service_network {
fail("The service_network for this service is undefined. Can't configure TLS for the internal network.")
}
# NOTE(jaosorior): The key of the internal_certificates_specs hash must
# must match the convention haproxy-<network name> or else this
# will fail. Futherly, it must contain the path that we'll use under
# 'service_pem'.
$internal_cert_path = $internal_certificates_specs["haproxy-${service_network}"]['service_pem']
$internal_bind_opts = union($haproxy_listen_bind_param, ['ssl', 'crt', $internal_cert_path])
} else {
# If no internal cert is given, we still configure TLS for the internal
# network, however, we expect that the public certificate has appropriate
# subjectaltnames set.
$internal_bind_opts = union($haproxy_listen_bind_param, ['ssl', 'crt', $public_certificate])
}
# NOTE(jaosorior): If the internal_ip and the public_virtual_ip are the
# same, the first option takes precedence. Which is the case when network
# isolation is not enabled. This is not a problem as both options are
# identical. If network isolation is enabled, this works correctly and
# will add a TLS binding to both the internal_ip and the
# public_virtual_ip.
# Even though for the public_virtual_ip the port 80 is listening, we
# redirect to https in the horizon_options below.
$horizon_bind_opts = {
Allow custom per-service bind_options for haproxy There are situation when it might be required to use different TLS versions between the services. HAproxy configures TLS version on bind line in the configuration, there is missing customization. At the moment we can only set TLS version globally via ssl_options This code's idea it to configure it per-service. For example, with: parameter_defaults: ExtraConfig: tripleo::haproxy::cinder::internal_bind_options: 'force-tlsv11' tripleo::haproxy::keystone_public::public_bind_options: 'force-tlsv12' tripleo::haproxy::horizon::public_bind_options: 'force-tlsv11' tripleo::haproxy::horizon::internal_bind_options: 'force-tlsv12' We will get something like the following in the haproxy config: listen cinder bind 2620:52:0:13b8:5054:ff:fe3e:1:13776 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem bind fd00:fd00:fd00:2000::17:8776 transparent force-tlsv11 ... listen keystone_public bind 2620:52:0:13b8:5054:ff:fe3e:1:13000 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv12 bind fd00:fd00:fd00:2000::17:5000 transparent ... listen horizon bind 2620:52:0:13b8:5054:ff:fe3e:1:443 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv11 bind 2620:52:0:13b8:5054:ff:fe3e:1:80 transparent force-tlsv11 bind fd00:fd00:fd00:2000::17:443 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv12 bind fd00:fd00:fd00:2000::17:80 transparent force-tlsv12 ... The two {public,internal}_bind_options accept both strings and arrays of strings. Closes-Bug: #1829328 Change-Id: I4b724a515d729c2e8e0da9cb8f081b8325d51a6b
2019-05-18 14:16:47 +02:00
"${internal_ip}:80" => union($haproxy_listen_bind_param, $custom_bind_options_internal),
"${internal_ip}:443" => union($internal_bind_opts, $custom_bind_options_internal),
"${public_virtual_ip}:80" => union($haproxy_listen_bind_param, $custom_bind_options_public),
"${public_virtual_ip}:443" => union($haproxy_listen_bind_param, ['ssl', 'crt', $public_certificate], $custom_bind_options_public),
}
Allow custom per-service listen_options for haproxy There are situations where it would be advantageous to let an operator specify custom per-service options. One such use case seen in the wild is to extend the timeout of the cinder because due to the specific storage backend these cinder operations sometimes take a bit longer. Letting the user tweak the haproxy_default_timeout is likely not what we want as for the case above we only want to tweak a single service. We explored another approach to fix this by adding a bunch of <service>_options class parameters in the tripleo::haproxy class but it made it extremely bloated and confusing, so we opted for this approach which is much less invasive both code-wise and complexity-wise Tested by deploying with: ExtraConfig: tripleo::haproxy::cinder::options: "timeout client": '90m' 'timeout server': '90m' And observing the following cinder haproxy stanza: listen cinder bind 10.0.0.4:8776 transparent bind 172.16.2.9:8776 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } option httpchk option httplog timeout client 90m timeout server 90m server overcloud-controller-0.internalapi.localdomain 172.16.2.7:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.16.2.16:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.16.2.13:8776 check fall 5 inter 2000 rise 2 Closes-Bug: #1755711 Change-Id: Icb7f026190b310d34c47dc059e2fdb22031b0963
2018-03-14 08:31:40 +01:00
$horizon_options = merge({
'cookie' => 'SERVERID insert indirect nocache',
'rsprep' => '^Location:\ http://(.*) Location:\ https://\1',
# NOTE(jaosorior): We always redirect to https for the public_virtual_ip.
'redirect' => 'scheme https code 301 if !{ ssl_fc }',
'option' => [ 'forwardfor', 'httpchk' ],
'http-request' => [
'set-header X-Forwarded-Proto https if { ssl_fc }',
'set-header X-Forwarded-Proto http if !{ ssl_fc }'],
Allow custom per-service listen_options for haproxy There are situations where it would be advantageous to let an operator specify custom per-service options. One such use case seen in the wild is to extend the timeout of the cinder because due to the specific storage backend these cinder operations sometimes take a bit longer. Letting the user tweak the haproxy_default_timeout is likely not what we want as for the case above we only want to tweak a single service. We explored another approach to fix this by adding a bunch of <service>_options class parameters in the tripleo::haproxy class but it made it extremely bloated and confusing, so we opted for this approach which is much less invasive both code-wise and complexity-wise Tested by deploying with: ExtraConfig: tripleo::haproxy::cinder::options: "timeout client": '90m' 'timeout server': '90m' And observing the following cinder haproxy stanza: listen cinder bind 10.0.0.4:8776 transparent bind 172.16.2.9:8776 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } option httpchk option httplog timeout client 90m timeout server 90m server overcloud-controller-0.internalapi.localdomain 172.16.2.7:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.16.2.16:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.16.2.13:8776 check fall 5 inter 2000 rise 2 Closes-Bug: #1755711 Change-Id: Icb7f026190b310d34c47dc059e2fdb22031b0963
2018-03-14 08:31:40 +01:00
}, $custom_options)
} else {
$horizon_bind_opts = {
Allow custom per-service bind_options for haproxy There are situation when it might be required to use different TLS versions between the services. HAproxy configures TLS version on bind line in the configuration, there is missing customization. At the moment we can only set TLS version globally via ssl_options This code's idea it to configure it per-service. For example, with: parameter_defaults: ExtraConfig: tripleo::haproxy::cinder::internal_bind_options: 'force-tlsv11' tripleo::haproxy::keystone_public::public_bind_options: 'force-tlsv12' tripleo::haproxy::horizon::public_bind_options: 'force-tlsv11' tripleo::haproxy::horizon::internal_bind_options: 'force-tlsv12' We will get something like the following in the haproxy config: listen cinder bind 2620:52:0:13b8:5054:ff:fe3e:1:13776 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem bind fd00:fd00:fd00:2000::17:8776 transparent force-tlsv11 ... listen keystone_public bind 2620:52:0:13b8:5054:ff:fe3e:1:13000 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv12 bind fd00:fd00:fd00:2000::17:5000 transparent ... listen horizon bind 2620:52:0:13b8:5054:ff:fe3e:1:443 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv11 bind 2620:52:0:13b8:5054:ff:fe3e:1:80 transparent force-tlsv11 bind fd00:fd00:fd00:2000::17:443 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem force-tlsv12 bind fd00:fd00:fd00:2000::17:80 transparent force-tlsv12 ... The two {public,internal}_bind_options accept both strings and arrays of strings. Closes-Bug: #1829328 Change-Id: I4b724a515d729c2e8e0da9cb8f081b8325d51a6b
2019-05-18 14:16:47 +02:00
"${internal_ip}:80" => union($haproxy_listen_bind_param, $custom_bind_options_internal),
"${public_virtual_ip}:80" => union($haproxy_listen_bind_param, $custom_bind_options_public),
}
Allow custom per-service listen_options for haproxy There are situations where it would be advantageous to let an operator specify custom per-service options. One such use case seen in the wild is to extend the timeout of the cinder because due to the specific storage backend these cinder operations sometimes take a bit longer. Letting the user tweak the haproxy_default_timeout is likely not what we want as for the case above we only want to tweak a single service. We explored another approach to fix this by adding a bunch of <service>_options class parameters in the tripleo::haproxy class but it made it extremely bloated and confusing, so we opted for this approach which is much less invasive both code-wise and complexity-wise Tested by deploying with: ExtraConfig: tripleo::haproxy::cinder::options: "timeout client": '90m' 'timeout server': '90m' And observing the following cinder haproxy stanza: listen cinder bind 10.0.0.4:8776 transparent bind 172.16.2.9:8776 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } option httpchk option httplog timeout client 90m timeout server 90m server overcloud-controller-0.internalapi.localdomain 172.16.2.7:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.16.2.16:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.16.2.13:8776 check fall 5 inter 2000 rise 2 Closes-Bug: #1755711 Change-Id: Icb7f026190b310d34c47dc059e2fdb22031b0963
2018-03-14 08:31:40 +01:00
$horizon_options = merge({
'cookie' => 'SERVERID insert indirect nocache',
'option' => [ 'forwardfor', 'httpchk' ],
Allow custom per-service listen_options for haproxy There are situations where it would be advantageous to let an operator specify custom per-service options. One such use case seen in the wild is to extend the timeout of the cinder because due to the specific storage backend these cinder operations sometimes take a bit longer. Letting the user tweak the haproxy_default_timeout is likely not what we want as for the case above we only want to tweak a single service. We explored another approach to fix this by adding a bunch of <service>_options class parameters in the tripleo::haproxy class but it made it extremely bloated and confusing, so we opted for this approach which is much less invasive both code-wise and complexity-wise Tested by deploying with: ExtraConfig: tripleo::haproxy::cinder::options: "timeout client": '90m' 'timeout server': '90m' And observing the following cinder haproxy stanza: listen cinder bind 10.0.0.4:8776 transparent bind 172.16.2.9:8776 transparent mode http http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto http if !{ ssl_fc } option httpchk option httplog timeout client 90m timeout server 90m server overcloud-controller-0.internalapi.localdomain 172.16.2.7:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.16.2.16:8776 check fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.16.2.13:8776 check fall 5 inter 2000 rise 2 Closes-Bug: #1755711 Change-Id: Icb7f026190b310d34c47dc059e2fdb22031b0963
2018-03-14 08:31:40 +01:00
}, $custom_options)
}
if $use_internal_certificates {
# Use SSL port if TLS in the internal network is enabled.
$backend_port = '443'
} else {
$backend_port = '80'
}
haproxy::listen { 'horizon':
bind => $horizon_bind_opts,
options => $horizon_options,
mode => 'http',
collect_exported => false,
}
Give horizon's stanza in haproxy a per-server cookie Currently our haproxy.cfg stanza for horizon looks like the following: server overcloud-controller-0.internalapi.localdomain 172.17.0.22:80 check cookie overcloud-controller-0 fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.17.0.25:80 check cookie overcloud-controller-0 fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.17.0.12:80 check cookie overcloud-controller-0 fall 5 inter 2000 rise 2 We need to make sure that the cookie is set the same as the server: server overcloud-controller-0.internalapi.localdomain 172.17.0.22:80 check cookie overcloud-controller-0.internalapi.localdomain fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.17.0.25:80 check cookie overcloud-controller-1.internalapi.localdomain fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.17.0.12:80 check cookie overcloud-controller-2.internalapi.localdomain fall 5 inter 2000 rise 2 The problem here is that the cookie is being inserted into the response by haproxy so that we have session persistence. When logging to horizon we want the session to be persistent and go to the same backend server. When haproxy sees a match for the cookie, it does just that. The cookie value will should match the server name. Prior to this fix each server was matching on the same cookie ($::hostname) which is not correct. Tested by connecting to horizon's VIP and shutting off horizon on each controller node one at the time. Observed that after each stop, the correct cookie from the remaining servers was sent to the browser. Closes-Bug: #1738453 Change-Id: Ieb9cf3c6a8373df288a73ff2dacfc9d0b09e675a
2017-09-11 21:45:15 +02:00
hash(zip($ip_addresses, $server_names)).each | $ip, $server | {
# We need to be sure the IP (IPv6) don't have colons
# which is a reserved character to reference manifests
$non_colon_ip = regsubst($ip, ':', '-', 'G')
haproxy::balancermember { "horizon_${non_colon_ip}_${server}":
Give horizon's stanza in haproxy a per-server cookie Currently our haproxy.cfg stanza for horizon looks like the following: server overcloud-controller-0.internalapi.localdomain 172.17.0.22:80 check cookie overcloud-controller-0 fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.17.0.25:80 check cookie overcloud-controller-0 fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.17.0.12:80 check cookie overcloud-controller-0 fall 5 inter 2000 rise 2 We need to make sure that the cookie is set the same as the server: server overcloud-controller-0.internalapi.localdomain 172.17.0.22:80 check cookie overcloud-controller-0.internalapi.localdomain fall 5 inter 2000 rise 2 server overcloud-controller-1.internalapi.localdomain 172.17.0.25:80 check cookie overcloud-controller-1.internalapi.localdomain fall 5 inter 2000 rise 2 server overcloud-controller-2.internalapi.localdomain 172.17.0.12:80 check cookie overcloud-controller-2.internalapi.localdomain fall 5 inter 2000 rise 2 The problem here is that the cookie is being inserted into the response by haproxy so that we have session persistence. When logging to horizon we want the session to be persistent and go to the same backend server. When haproxy sees a match for the cookie, it does just that. The cookie value will should match the server name. Prior to this fix each server was matching on the same cookie ($::hostname) which is not correct. Tested by connecting to horizon's VIP and shutting off horizon on each controller node one at the time. Observed that after each stop, the correct cookie from the remaining servers was sent to the browser. Closes-Bug: #1738453 Change-Id: Ieb9cf3c6a8373df288a73ff2dacfc9d0b09e675a
2017-09-11 21:45:15 +02:00
listening_service => 'horizon',
ports => $backend_port,
ipaddresses => $ip,
server_names => $server,
options => union($member_options, ["cookie ${server}"]),
}
}
Fix horizon firewall rules in composable roles Atm horizon haproxy firewall rules obfuscate any other rule defined via the tripleo.haproxy.firewall_rules key. Things broke with https://review.opendev.org/#/c/625600/. The reason that was pushed is that in composable roles, when splitting off horizon away from where haproxy runs, we would not have the proper iptables rules on the haproxy role. This was due to the fact that we had the following code: service_config_settings: haproxy: tripleo.horizon.firewall_rules: '127 horizon': dport: - 80 - 443 The above code never worked as explained in 3f8ce6fd96bc4f28a052b4c87a19b4b152734091 and so we fixed it by setting the proper tripleo.haproxy.firewall_rules key. The issue is that rules for haproxy should just never have been set at all via service_config_settings keys in the first place. As demonstrated with this bug, the merging of hiera dictionaries will mess us up and we'll end up overwriting other keys. Haproxy stats access has this: outputs: role_data: description: Role data for the HAproxy role. value: service_name: haproxy monitoring_subscription: {get_param: MonitoringSubscriptionHaproxy} config_settings: map_merge: - tripleo.haproxy.firewall_rules: '107 haproxy stats': dport: 1993 And since hiera will return the horizon settings for tripleo.haproxy.firewall_rules which won't be deep merged with the firewall rules from haproxy stats and so rule '107 haproxy stats' will never be present. Rules for haproxy need to happen in puppet-tripleo/manifests/haproxy*. Normally they do, the exception is horizon which uses a specialized horizon_endpoint.pp manifest which does not trigger these rules. Let's create the firewall rules in haproxy/horizon_endpoint.pp like we do for all other endpoints. Tested and correctly got: [root@controller-0 ~]# iptables -nvL |grep hor 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 80 state NEW /* 100 horizon_haproxy ipv4 */ 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 443 state NEW /* 100 horizon_haproxy_ssl ipv4 */ 0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 80,443 state NEW /* 126 horizon ipv4 */ Change-Id: I1325171ef60d7a7e3b57373082fcdb5487be939b Related-Bug: #1829338
2019-05-16 09:12:50 +02:00
if $manage_firewall {
include ::tripleo::firewall
$haproxy_horizon_firewall_rules = {
'100 horizon_haproxy' => {
'dport' => 80,
},
}
if $public_certificate {
$haproxy_horizon_ssl_firewall_rules = {
'100 horizon_haproxy_ssl' => {
'dport' => 443,
},
}
} else {
$haproxy_horizon_ssl_firewall_rules = {}
}
$horizon_firewall_rules = merge($haproxy_horizon_firewall_rules, $haproxy_horizon_ssl_firewall_rules)
if !empty($horizon_firewall_rules) {
create_resources('tripleo::firewall::rule', $horizon_firewall_rules)
}
}
}