opendev/system-config
Code Issues Proposed changes

Split logstash gearman from elasticsearch fw rules

Browse Source 
We were using the list of elasticsearch clients to generate firewall
rules for both elasticsearch port ranges and gearman port ranges. We
really only need subsets of the super set in both cases so lets make two
distinct lists that we can add and remove servers from instead.

This means the two sets of nodes that can talk to elasticsaerch are the
logstash node for kibana and the logstash workers for indexing. Then
all zuul executors and logstash workers and subunit workers can talk to
logstash.o.o over port 4730 for gearman job submission and handling.

Change-Id: I95de1404dcc087f09f0fd4e4134e20673e8c0ae5
This commit is contained in:
Clark Boylan 2017-12-13 13:25:47 -08:00
parent ed56c69e12
commit 32c991cad2
2 changed files with 23 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
hiera/common.yaml
View File

@ -28,6 +28,27 @@ elasticsearch_clients:
- logstash-worker18.openstack.org
- logstash-worker19.openstack.org
- logstash-worker20.openstack.org
logstash_gearman_clients:
- logstash-worker01.openstack.org
- logstash-worker02.openstack.org
- logstash-worker03.openstack.org
- logstash-worker04.openstack.org
- logstash-worker05.openstack.org
- logstash-worker06.openstack.org
- logstash-worker07.openstack.org
- logstash-worker08.openstack.org
- logstash-worker09.openstack.org
- logstash-worker10.openstack.org
- logstash-worker11.openstack.org
- logstash-worker12.openstack.org
- logstash-worker13.openstack.org
- logstash-worker14.openstack.org
- logstash-worker15.openstack.org
- logstash-worker16.openstack.org
- logstash-worker17.openstack.org
- logstash-worker18.openstack.org
- logstash-worker19.openstack.org
- logstash-worker20.openstack.org
- subunit-worker01.openstack.org
- subunit-worker02.openstack.org
- ze01.openstack.org

3
manifests/site.pp
View File

@ -7,6 +7,7 @@
#
$elasticsearch_nodes = hiera_array('elasticsearch_nodes')
$elasticsearch_clients = hiera_array('elasticsearch_clients')
$logstash_gearman_clients = hiera_array('logstash_gearman_clients')
#
# Default: should at least behave like an openstack server
@ -468,7 +469,7 @@ node /^wiki-dev\d+\.openstack\.org$/ {
node /^logstash\d*\.openstack\.org$/ {
$iptables_es_rule = regsubst($elasticsearch_nodes,
'^(.*)$', '-m state --state NEW -m tcp -p tcp --dport 9200:9400 -s \1 -j ACCEPT')
$iptables_gm_rule = regsubst($elasticsearch_clients,
$iptables_gm_rule = regsubst($logstash_gearman_clients,
'^(.*)$', '-m state --state NEW -m tcp -p tcp --dport 4730 -s \1 -j ACCEPT')
$logstash_iptables_rule = flatten([$iptables_es_rule, $iptables_gm_rule])