12975 Commits

Author SHA1 Message Date
Ian Wienand
de79f06585 Put hound sync lock in /var/run
/var/run/hound isn't a directory, so this fails.  It was supposed to
just be in /var/run.

Also, we don't want to run it every minute of 4am ...

Change-Id: I7298a0e18a63bf8331686bd4c44e3a12b9c77176
2017-12-18 15:14:07 +11:00
Zuul
b98a281adb Merge "Add kerberos / afs dns info" 2017-12-15 17:59:42 +00:00
Zuul
b66a25650f Merge "Add kdc04.o.o xenial node" 2017-12-15 17:59:41 +00:00
Zuul
c449cf0335 Merge "Clean up openstack_project::server for kdc01 / kdc02" 2017-12-15 17:54:13 +00:00
Paul Belanger
17777fc901
Add kdc04.o.o xenial node
Bring online

Change-Id: I52fea922914cb8b9fbc02a839ff520ddfe58e93a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-15 11:40:13 -05:00
James E. Blair
faa31fa404 Add kerberos / afs dns info
Change-Id: Id2cc43f1d67584ac26709d61679b3c6659df8daa
2017-12-15 08:24:26 -08:00
Zuul
524c7b404c Merge "Delete design-summit-prep node" 2017-12-15 16:14:59 +00:00
Zuul
547db911c2 Merge "Clean up cacti.o.o node settings" 2017-12-15 16:09:23 +00:00
Paul Belanger
0c09b73e13
Clean up openstack_project::server for kdc01 / kdc02
Move openstack_project::server into site.pp like other nodes, this was
the old way of provisioning servers.

Change-Id: If36ace9c377881e25d30e1f7f0184383b894ca17
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-15 11:01:06 -05:00
Paul Belanger
3f7f84cb38
Delete design-summit-prep node
We no longer use this server or service, so we can delete it.

Change-Id: Iaad4ad9f0517dba86ea3ee78d08b0904f621c818
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-15 10:16:32 -05:00
Jens Harbott
8b445237ea Prepare for replacing elasticsearch02
logstash uses the first of the discover_nodes as proxy, so we need to
rotate the list in order to be able to replace the first node without
impacting service.

Also replace the discover_node for the logstash-workers accordingly.

Change-Id: Ib6f6d19a766021f9f16fb3bc2de1d80df66f7671
2017-12-15 15:10:41 +00:00
Zuul
154876b422 Merge "Add dns servers" 2017-12-15 14:51:47 +00:00
Zuul
2631f10226 Merge "Update elasticsearch firewall rules" 2017-12-15 09:20:25 +00:00
Zuul
26ccab85f1 Merge "xenial update for codesearch.o.o" 2017-12-15 01:48:14 +00:00
Zuul
65afee9095 Merge "Add javascript alias to cacti.o.o for xenial" 2017-12-15 01:30:58 +00:00
James E. Blair
83ba1311df Add dns servers
Change-Id: I32b0d846cbbaad5755d3d1c47d303b7cdf34f749
Depends-On: Ic92726dc341af5802ad803d239bd547ef5068043
Story: 2001382
Task: 6090
2017-12-14 17:04:54 -08:00
Zuul
17f0987e6b Merge "Remove npm mirroring components" 2017-12-15 00:12:01 +00:00
Paul Belanger
e711642984
Clean up cacti.o.o node settings
Now that we have migrated to ubuntu-xenial, we can stop testing on
trusty. We can also clean out old cacti.o.o and cacti01.o.o firewall
rules from our base server.pp.

Change-Id: I84b96de40a79d8103cfce5ec121e13a7d01f729d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-14 19:08:14 -05:00
Zuul
e18299d605 Merge "Update logstash gearman client firewall rules" 2017-12-15 00:05:25 +00:00
Paul Belanger
46750e896a
Add javascript alias to cacti.o.o for xenial
It seems there have been some changes to how javascript libraries are
installed in ubuntu xenial. Add an alias to /usr/share/javascript.

Change-Id: I1ea75cd5c9fddc04515414427f9f322d83f14ecb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-14 19:03:42 -05:00
Zuul
aeb6dec058 Merge "Add cacti02.o.o to all snmp iptables rules" 2017-12-14 23:51:17 +00:00
Clark Boylan
e99506b030 Update elasticsearch firewall rules
This converts the config for elasticsearch cluster client firewall rules
to use the new puppet-iptables iptables_allowed_hosts feature. This works
around an issue with netfilter-persistent starting before dns
resolution is working on boot.

Change-Id: I81b7598cb32d498b219ee00f0589e6bf0dc8c242
2017-12-14 15:49:38 -08:00
Ian Wienand
d7a56cff63 xenial update for codesearch.o.o
Change-Id: I7859eb26168e5ee11ab2290e55409ff6b86aceab
2017-12-15 10:32:06 +11:00
Ian Wienand
7ace3799f1 Remove npm mirroring components
The npm mirror was removed with
Id539d336814cce2ce18898526e561b8b6977f62f.  This change is "inspired"
by a proposed puppet-nodejs update in
Ia7966fb9578d0d79f3a7f9480e3a956555737dc8.  Rather than fixing it up
for the new version, remove it (also, puppet is failing trying to
access /afs/.openstack.org/mirror/npm).

I believe the npmrc.erb file is actually a vestige of prior release
methods and is also no longer required.

Change-Id: I6fa48e4700779d2c90194f0129c770bf2d6d865f
2017-12-15 10:25:18 +11:00
Paul Belanger
3bdbe3b7f3
Add cacti02.o.o to all snmp iptables rules
As we upgrade cacti02.o.o to xenial, we need to allow it access to all
servers to collect stats.  We can delete old firewall rules in a
follow patch.

Change-Id: I0bbd3e82fdf8644159dfe82b1dfc5478ef5095bb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-14 18:09:34 -05:00
Zuul
759c3db50a Merge "Simplify elasticsearch firewall rules" 2017-12-14 23:07:10 +00:00
Zuul
96b3917a2a Merge "Use iptables allowed_hosts" 2017-12-14 23:07:00 +00:00
Zuul
7f9adeb572 Merge "Fix another cacti dependency issue with /var/lib/cacti" 2017-12-14 22:23:22 +00:00
Clark Boylan
6f4637c1ef Update logstash gearman client firewall rules
This converts the config for logstsah gearman client firewall rules to
use the new puppet-iptables iptables_allowed_hosts feature. This works
around an issue with netfilter-persistent starting before dns
resolution is working on boot.

Change-Id: I76c45d8edbfe9f5420884e0ef2fb62cff2cc2bc9
2017-12-14 14:08:23 -08:00
Clark Boylan
5f876310ce Simplify elasticsearch firewall rules
Because we are no longer running elasticsearch daemons on
logstash-workers to perform indexing (and instead use http to the
elasticsearch cluster data nodes) and because kibana also speaks the
http API and doesn't join the cluster from logstash.openstack.org we
don't need to allow the full mesh of connectivity over ports 9200 to
9400.

Remove these unneeded firewall rules as the next step is converting to
the new dns resolving firewall rule builder parameter in
puppet-iptables.

Change-Id: If79bab6dc0b510c5589b83c943458e8580eb8092
2017-12-14 13:44:16 -08:00
Paul Belanger
a376c9db5a Fix another cacti dependency issue with /var/lib/cacti
Seem there is a race on xenial where we try to populate /var/lib/cacti
before it is created.

Change-Id: I179e2e2d9d4f9df53aace172950af66aed92efad
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-14 21:42:24 +00:00
James E. Blair
994365d1bf Use iptables allowed_hosts
This allows us to more safely specify hosts by name in iptable
rules, as they will be resolved by puppet before being written
to disk.

Change-Id: Ie133ad8246d5907723a6d7cbf14644e0a10cc4e7
Depends-On: I7a0dfbab67bdba72c0a56acc611503795d2bc350
2017-12-14 11:16:42 -08:00
James E. Blair
0a57d5156c Add dnsquery puppet module
Change-Id: I29d36cc527351e3e6d2ee2dc1919988379b8db3a
2017-12-14 11:16:14 -08:00
Paul Belanger
a304c5a718
Fix dependency issue with cacti package
Right now the cacti package setups up the /usr/local/share/cacti
directories. So we need to make sure cacti is installed before we
start adding files into those directories.

Change-Id: I99bbf0a71e140380636419c4200d9f4662f5311e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-14 11:32:54 -05:00
Zuul
671a12cd3c Merge "Update cacti.o.o testing to xenial" 2017-12-14 13:35:01 +00:00
Zuul
3bbc2be599 Merge "Fixes for codesearch resync" 2017-12-14 04:05:13 +00:00
Zuul
0c30efa777 Merge "Bump puppetlabs-apache to 1.11.1" 2017-12-13 22:57:51 +00:00
Ian Wienand
795bf1c971 Fixes for codesearch resync
This provides a script to resync the hound configuration for
codesearch.

It checks if the config needs updating, and if so, puts the new config
file in place, takes the reindex lock from
I9d28201bca75b624e07cbba14c870151094fc7ae and restarts the service,
waiting until it sees it is up.

puppet is currently disabled on this host because updates are tied to
project-config changes; since restarting takes the service down this
is obviously not great.

Change-Id: Iaebf50836607da447dcf1765ec01d0121537b0da
Depends-On: I9d28201bca75b624e07cbba14c870151094fc7ae
2017-12-14 09:48:01 +11:00
Paul Belanger
6f1e240868
Update cacti.o.o testing to xenial
Change-Id: Idaba4705ffdaea1cb2d7da07db6752a9a1162907
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-13 17:05:26 -05:00
Paul Belanger
fa7c94f150
Bump puppetlabs-apache to 1.11.1
This is currently the last 1.x release that still support puppet 3, we
could make the jobs to puppet 4, but for now we just need a new enough
module for xenial support.

Also, seems puppetlabs-apache is only uesd by cacti.o.o.

Change-Id: I128a0d8d851311b77592d98ded5891d71dce2031
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-13 16:44:15 -05:00
Clark Boylan
32c991cad2 Split logstash gearman from elasticsearch fw rules
We were using the list of elasticsearch clients to generate firewall
rules for both elasticsearch port ranges and gearman port ranges. We
really only need subsets of the super set in both cases so lets make two
distinct lists that we can add and remove servers from instead.

This means the two sets of nodes that can talk to elasticsaerch are the
logstash node for kibana and the logstash workers for indexing. Then
all zuul executors and logstash workers and subunit workers can talk to
logstash.o.o over port 4730 for gearman job submission and handling.

Change-Id: I95de1404dcc087f09f0fd4e4134e20673e8c0ae5
2017-12-13 13:25:47 -08:00
Zuul
ed56c69e12 Merge "Revert "Remove subunit-worker01.openstack.org"" 2017-12-13 19:27:55 +00:00
Zuul
8ac334eca9 Merge "Retire apps site remains" 2017-12-13 18:36:07 +00:00
Jeremy Stanley
d127c069d3 Revert "Remove subunit-worker01.openstack.org"
The server has been rebuilt on Ubuntu Xenial and is ready to go back
into service.

This reverts commit 664689e42729fdbc750ee74f481687cb4d9ee3f0.

Change-Id: I3e7a388fc01d99c5534ace678864dd5840f8e6d8
2017-12-13 15:40:49 +00:00
Zuul
649d5a0310 Merge "Reduce cacti updates cron spam" 2017-12-13 00:48:25 +00:00
Zuul
dfef5c70dc Merge "Create paste hiera group" 2017-12-13 00:39:59 +00:00
Zuul
5ffa691503 Merge "Support xenial on subunit-worker" 2017-12-12 23:39:47 +00:00
James E. Blair
fe92742558 Create paste hiera group
Change-Id: I0a9fd98c7a95bdf7046c5a56bbbd0fab3f27412e
2017-12-12 14:33:23 -08:00
Clark Boylan
0293b260d9 Reduce cacti updates cron spam
The current process to udpate cacti hosts is fairly chatty. It basically
goes in and just tries to update every host there and if they exist that
generates a bunch of output which is then emailed to infra roots. This
information is potentially useful for debugging so keep it around in
local cacti host logs. These logs will then be rotated with a week of
retention.

This should help make our inboxes happier.

Change-Id: Ib03ef7b22083a2a2454715bd5229313b19b84ae9
2017-12-12 14:00:08 -08:00
Zuul
ea950aadeb Merge "Support xenial on lists" 2017-12-12 21:41:48 +00:00