70 Commits

Author SHA1 Message Date
Zuul
f5b19b4598 Merge "Turn on the future parser for elasticsearch.openstack.org" 2018-12-18 20:17:52 +00:00
Zuul
854f3ba29f Merge "Turn on the future parser for subunit workers" 2018-12-18 01:00:43 +00:00
Zuul
6b96f59405 Merge "Turn on the future parser for logstash.openstack.org" 2018-12-14 20:00:31 +00:00
Colleen Murphy
100173dfe2 Turn on the future parser for elasticsearch.openstack.org
Change-Id: I91b5a7d15fa29ad1e604a5cf3a1bd32dfa5fa892
2018-12-13 15:26:12 -08:00
Colleen Murphy
7313bd0bb2 Turn on the future parser for subunit workers
Change-Id: I3308b015e99442c48f8a044a30ba50989eb9647c
2018-12-13 15:26:04 -08:00
Colleen Murphy
c5901672fa Turn on the future parser for logstash.openstack.org
Change-Id: I9f8056b91acf413f1d541b89aea5b98ca0cdfd1e
2018-12-13 15:25:59 -08:00
Zuul
19ce2f6292 Merge "Turn on the future parser for wiki.openstack.org" 2018-12-13 23:20:14 +00:00
Zuul
04b2ee6e18 Merge "Turn on the future parser for wiki-dev.openstack.org" 2018-12-13 23:20:11 +00:00
Zuul
0b327f3ab1 Merge "Turn on the future parser for lists.openstack.org" 2018-12-12 19:53:22 +00:00
James E. Blair
7f3963efca Add ze12.openstack.org
We believe the relative_priority change has altered our workload
such that we have smaller jobs starting more frequently.  Since
job starts are limited by the executors, we have developed a backlog
and need another executor to relieve the pressure.

Change-Id: I98052e0135c7ee615f1f187b9d0a250cdd1ff178
2018-12-05 14:08:17 -08:00
Zuul
60393c746c Merge "Turn on future parser for lists.katacontainers.io" 2018-12-03 22:19:23 +00:00
Monty Taylor
1a7dedd13f
Switch to a static inventory
Rather than be beholden to a dynamic multi-cloud process for data that
is mostly static, just pre-generated it and check it in to git. This
will remove the need to run the openstack dynamic inventory at all.

Change-Id: Ib5c38e9addc55201fd68cbe5f09684391686904e
2018-11-29 15:44:06 -06:00
Colleen Murphy
8498fb0988 Turn on the future parser for wiki.openstack.org
Change-Id: Ie6de2e78b4f8292e9837f628f75ae229637d4c91
2018-11-06 21:27:11 +01:00
Colleen Murphy
f3a73fa03f Turn on the future parser for wiki-dev.openstack.org
Change-Id: Id78b81c24b9ac57af27082771146aca7072c0eb6
2018-11-05 22:21:39 +01:00
Colleen Murphy
e9bcd65a5b Turn on the future parser for lists.openstack.org
Change-Id: I9481f1b7cbd0eb1a266e2c0d9ba9d3c784bab737
2018-11-05 22:20:48 +01:00
Colleen Murphy
b45e642f97 Turn on future parser for lists.katacontainers.io
The mailman puppet module is tested and seems to be working on puppet
4[1]. Turn on the future parser for lists.katacontainers.io, which hosts
fewer mailing lists than lists.openstack.org does.

[1] https://review.openstack.org/601464

Change-Id: Iea7fbabf7cee579dcd19d4deb4d7537ed1b13bd9
Depends-On: https://review.openstack.org/601831
2018-11-05 22:18:26 +01:00
Colleen Murphy
e9576c0677 Turn on the future parser for eavesdrop.o.o
eavesdrop.o.o runs a number of different sites and bots, so on the one
hand it is a slightly complex service, but on the other hand these
modules are quite simple and similar to each other. Meetbot, accessbot,
and statusbot have tests and seem to work fine on puppet 4. Ptgbot does
not have tests, but that's why we should try this out sooner rather than
later so we have time to correct any issues before the PTG.

Change-Id: Id1af4b3ebf04f34c381eab5f818815734a296f7b
2018-11-05 22:18:22 +01:00
James E. Blair
90e6088881 Configure adns1.opendev.org server via ansible
Change-Id: Ib4d3cd7501a276bff62e3bc0998d93c41f3ab185
2018-11-02 13:49:38 -07:00
Clark Boylan
ab5c1944cc Simplify elasticsearch cluster groups.yaml membership
We run a cluster of elasticsearch nodes which may have additional
servers added to it or old servers replaced. Make this slightly easier
by matching them with a glob in groups.yaml so that we don't have to
edit this file to add new servers.

Change-Id: I8ce3e38f67c545642d561b2afdd6dbbfa677e67f
2018-11-02 12:20:20 -07:00
Clark Boylan
47bf937f8d Simplify git backend server groups.yaml membership
We run many git server backend nodes which may be replaced with new
servers. To simplify the process of adding or replacing git0* servers
stop explicitly listing them in groups.yaml and instead match them on a
glob.

Change-Id: I3dbf814122bbfc508b2a90a5f5a533226fa0bf8f
2018-11-02 12:20:20 -07:00
Clark Boylan
7427cc7a98 Simplify zookeeper groups.yaml membership
We run a cluster of zookeeper nodes. Lets match them with globs in
groups.yaml so that we don't have to edit groups.yaml if expanding the
size ofthe cluster or replacing nodes in the cluster.

Change-Id: Iff9048a9bc15e4f888e7caab54abb4d4a9d340d7
2018-11-02 12:20:20 -07:00
Clark Boylan
b53a03df68 Simplify nodepool groups.yaml membership
Replace all the explicit listings of nb01, nl01, etc with globs so that
we can launch new nodepool builders and nodepool launchers without
editing the groups file.

Change-Id: I3c32fb604109a3c65b2145b733fc0e2545e21045
2018-11-02 12:20:20 -07:00
Clark Boylan
9beb96838b Simplify mirror node listings in groups.yaml
Match the mirrors with a glob so that we can boot new mirrors in new
regions or replacements in existing regions without needing to
explicitly add them to groups.yaml first.

Change-Id: I3e7bf04a2d3a7602e46710840b0cec42c238c6c1
2018-11-02 12:20:20 -07:00
Clark Boylan
1947bfc3ef Simpligy logstash nodes in groups.yaml
Match logstash-worker and logstash.o.o nodes with globs. This allows us
to launch new servers without needing to add them to the groups file
first.

Note that logstash.o.o is currently hosted by logstash01.openstack.org
so the existing groups.yaml content is broken for this host.

Change-Id: Iea9b5815a31355925979e6d4da5c578a64af7972
2018-11-02 12:20:20 -07:00
Clark Boylan
5a98029ccc Cleanup zuul-executors in groups.yaml
Convert ze01, ze02, ze03, etc to ze[0-9]* so that we can launch new zuul
executors as necessary without updating the groups file.

Change-Id: Id9896e89d8bc821de003994a96ae11517a76145d
2018-11-02 12:20:20 -07:00
Clark Boylan
6273236b72 Cleanup zuul-mergers in groups.yaml
The zuul mergers do not run a webserver so we remove them from the
webserver group. Convert zm01, zm02, zm03, etc to zm[0-9]* so that we
can launch new zuul mergers as necessary without updating the groups
file.

Change-Id: Ibe8cbea02990378f4ee5ba7880ee0c8a4e9b7bb9
2018-11-02 12:20:20 -07:00
Jeremy Stanley
681574b352 Merge "Update etherpad group membership" 2018-11-02 18:12:01 +00:00
Jeremy Stanley
d874d4da63 Remove mirrors from the webservers group
The webservers group is meant to be mutually exclusive to any other
iptables modifications but was inadvertently expanded to include the
mirror servers by Iea8b2eb2e9c723aca06f75d3d3307893e320cced. Remove
the glob as it should not be there and is overriding the
mirror-specific firewall rules.

Change-Id: Ia1b702e5c9a2d433536edcec4bfaf6f47b1c0291
2018-11-02 18:02:43 +00:00
Clark Boylan
855e9cf5e0 Update etherpad group membership
Our etherpad servers now have digits so we need them to have digits in
the groups file. Also we have etherpad-dev group_vars that are different
than the etherpad group_vars values so add a group for etherpad-dev to
ensure the correct hiera data is used on the dev server.

Change-Id: I9e34ef7969b8a562192a29d98ead6fc55f71d5f2
2018-11-02 10:19:37 -07:00
Monty Taylor
e998db36f2 Add yamlgroup inventory plugin
The constructed inventory plugin allows expressing additional groups,
but it's too heavy weight for our needs. Additionally, it is a full
inventory plugin that will add hosts to the inventory if they don't
exist.

What we want instead is something that will associate existing hosts
(that would have come from another source) with groups.

This also switches to using emergency.yaml instead of emergency, which
uses the same format.

We add an extra groups file for gate testing to ensure the CI nodes
get puppet installed.

Change-Id: Iea8b2eb2e9c723aca06f75d3d3307893e320cced
2018-11-02 08:19:53 +11:00
Zuul
abece6272a Merge "Turn on the future parser for survey.o.o" 2018-10-15 17:07:31 +00:00
Zuul
d86c021fa1 Merge "Turn on the future parser for status.o.o" 2018-10-15 17:02:02 +00:00
Zuul
a7f9a784f8 Merge "Turn on the future parser for static.o.o" 2018-10-15 17:00:43 +00:00
Clark Boylan
0f0031d16e Add digit support to etherpad servers
Looking at upgrading etherpad servers to Xenial and realize that we
haven't "digitized" them yet. Lets get that in first so that the new
instances we boot can be etherpad01 and etherpad-dev01.

Change-Id: I700a1d018ab37436f11b2d883d79c82db7f1d6b1
2018-09-24 13:33:44 -07:00
Zuul
251661e6df Merge "Turn on the future parser for pbx.o.o" 2018-09-19 10:37:39 +00:00
Zuul
4c162e59a6 Merge "Turn on the future parser for cacti.o.o" 2018-09-18 22:44:03 +00:00
Zuul
56d9de1039 Merge "Turn on the future parser for translate.o.o" 2018-09-17 21:58:55 +00:00
Zuul
c66a5ffd80 Merge "Turn on the future parser for translate-dev" 2018-09-14 17:51:06 +00:00
Zuul
989608fbe0 Merge "Turn on the future parser for storyboard" 2018-09-13 21:50:46 +00:00
Clark Boylan
681e24d27d Treat unparsed inventory as a failure
We don't want to run ansible if we don't get a complete inventory from
our clouds. The reason for this is we cannot be sure that the ordering
of git servers, gerrit, and zuul or our serialized updates of afs
servers will work correctly if we have an incomplete inventory.

Instead we just want ansible to fail and try again in the future (we can
then debug why our clouds are not working).

From the ansible docs for any_unparsed_is_failed:

  If 'true', it is a fatal error when any given inventory source
  cannot be successfully parsed by any available inventory plugin;
  otherwise, this situation only attracts a warning.

Additionally we tell openstack inventory plugin to report failures
rather than empty inventory so that the unparsed failures happen.

Change-Id: I9025776af4316fbdd2c910566883eb3a2530852a
2018-09-12 15:09:21 -07:00
Colleen Murphy
002d0c33bc Revert "Revert "Turn on the future parser for storyboard-dev""
This reverts commit 701116dc9b05308b1fa7839590b6f16c5d71ae02.

Change-Id: I08ad7d55d0c88a9df51ac9d9716ae377a9a5d707
Depends-On: https://review.openstack.org/602092
2018-09-12 18:43:16 +02:00
Colleen Murphy
701116dc9b Revert "Turn on the future parser for storyboard-dev"
There is an ERB parsing error for puppet 4, roll back while we figure it
out.

This reverts commit a156accb7d2799cb76e263c927f6bcfaac3c22b2.

Change-Id: I1d2cd468a8d1ebab2573b03250bf6c3ba1195798
2018-09-12 17:52:00 +02:00
Colleen Murphy
f702d9c120 Turn on the future parser for survey.o.o
Change-Id: Id137f31cb11bd5ed8f75097fdc9225f0af0e1fb8
2018-09-12 00:13:35 +02:00
Colleen Murphy
232921dba2 Turn on the future parser for status.o.o
Change-Id: I24911373181a8957a4424f69b095bad67487ef34
2018-09-12 00:12:29 +02:00
Colleen Murphy
fbbb35deb5 Turn on the future parser for static.o.o
Change-Id: Ice5aa8c8299fae280b77a319ecab46a61ee7f767
2018-09-12 00:10:46 +02:00
Colleen Murphy
c32acacd18 Turn on the future parser for pbx.o.o
Change-Id: I9f0a73b0e638edfefa433bb670e219fb48c20f86
Depends-On: https://review.openstack.org/601749
2018-09-12 00:08:31 +02:00
Colleen Murphy
eccda01f74 Turn on the future parser for cacti.o.o
Change-Id: I98822d3524ad31fa3c888b08c92f6586fe1a87d5
2018-09-11 00:20:34 +02:00
Colleen Murphy
644d8ab502 Turn on the future parser for translate.o.o
Change-Id: I5a873aa62e72aa7713134d688eb52d1549a3139a
2018-09-11 00:20:34 +02:00
Colleen Murphy
0d4eff144a Turn on the future parser for translate-dev
Change-Id: I1b05def167c3336d7bce931a63d9336f54853ccb
2018-09-11 00:20:31 +02:00
Colleen Murphy
b2441cb3d5 Turn on the future parser for storyboard
Change-Id: I3db7e0954bdb56b8863d6aba993aa6f0563e333d
2018-09-11 00:20:10 +02:00