We believe the relative_priority change has altered our workload
such that we have smaller jobs starting more frequently. Since
job starts are limited by the executors, we have developed a backlog
and need another executor to relieve the pressure.
Change-Id: I98052e0135c7ee615f1f187b9d0a250cdd1ff178
Rather than be beholden to a dynamic multi-cloud process for data that
is mostly static, just pre-generated it and check it in to git. This
will remove the need to run the openstack dynamic inventory at all.
Change-Id: Ib5c38e9addc55201fd68cbe5f09684391686904e
The mailman puppet module is tested and seems to be working on puppet
4[1]. Turn on the future parser for lists.katacontainers.io, which hosts
fewer mailing lists than lists.openstack.org does.
[1] https://review.openstack.org/601464
Change-Id: Iea7fbabf7cee579dcd19d4deb4d7537ed1b13bd9
Depends-On: https://review.openstack.org/601831
eavesdrop.o.o runs a number of different sites and bots, so on the one
hand it is a slightly complex service, but on the other hand these
modules are quite simple and similar to each other. Meetbot, accessbot,
and statusbot have tests and seem to work fine on puppet 4. Ptgbot does
not have tests, but that's why we should try this out sooner rather than
later so we have time to correct any issues before the PTG.
Change-Id: Id1af4b3ebf04f34c381eab5f818815734a296f7b
We run a cluster of elasticsearch nodes which may have additional
servers added to it or old servers replaced. Make this slightly easier
by matching them with a glob in groups.yaml so that we don't have to
edit this file to add new servers.
Change-Id: I8ce3e38f67c545642d561b2afdd6dbbfa677e67f
We run many git server backend nodes which may be replaced with new
servers. To simplify the process of adding or replacing git0* servers
stop explicitly listing them in groups.yaml and instead match them on a
glob.
Change-Id: I3dbf814122bbfc508b2a90a5f5a533226fa0bf8f
We run a cluster of zookeeper nodes. Lets match them with globs in
groups.yaml so that we don't have to edit groups.yaml if expanding the
size ofthe cluster or replacing nodes in the cluster.
Change-Id: Iff9048a9bc15e4f888e7caab54abb4d4a9d340d7
Replace all the explicit listings of nb01, nl01, etc with globs so that
we can launch new nodepool builders and nodepool launchers without
editing the groups file.
Change-Id: I3c32fb604109a3c65b2145b733fc0e2545e21045
Match the mirrors with a glob so that we can boot new mirrors in new
regions or replacements in existing regions without needing to
explicitly add them to groups.yaml first.
Change-Id: I3e7bf04a2d3a7602e46710840b0cec42c238c6c1
Match logstash-worker and logstash.o.o nodes with globs. This allows us
to launch new servers without needing to add them to the groups file
first.
Note that logstash.o.o is currently hosted by logstash01.openstack.org
so the existing groups.yaml content is broken for this host.
Change-Id: Iea9b5815a31355925979e6d4da5c578a64af7972
Convert ze01, ze02, ze03, etc to ze[0-9]* so that we can launch new zuul
executors as necessary without updating the groups file.
Change-Id: Id9896e89d8bc821de003994a96ae11517a76145d
The zuul mergers do not run a webserver so we remove them from the
webserver group. Convert zm01, zm02, zm03, etc to zm[0-9]* so that we
can launch new zuul mergers as necessary without updating the groups
file.
Change-Id: Ibe8cbea02990378f4ee5ba7880ee0c8a4e9b7bb9
The webservers group is meant to be mutually exclusive to any other
iptables modifications but was inadvertently expanded to include the
mirror servers by Iea8b2eb2e9c723aca06f75d3d3307893e320cced. Remove
the glob as it should not be there and is overriding the
mirror-specific firewall rules.
Change-Id: Ia1b702e5c9a2d433536edcec4bfaf6f47b1c0291
Our etherpad servers now have digits so we need them to have digits in
the groups file. Also we have etherpad-dev group_vars that are different
than the etherpad group_vars values so add a group for etherpad-dev to
ensure the correct hiera data is used on the dev server.
Change-Id: I9e34ef7969b8a562192a29d98ead6fc55f71d5f2
The constructed inventory plugin allows expressing additional groups,
but it's too heavy weight for our needs. Additionally, it is a full
inventory plugin that will add hosts to the inventory if they don't
exist.
What we want instead is something that will associate existing hosts
(that would have come from another source) with groups.
This also switches to using emergency.yaml instead of emergency, which
uses the same format.
We add an extra groups file for gate testing to ensure the CI nodes
get puppet installed.
Change-Id: Iea8b2eb2e9c723aca06f75d3d3307893e320cced
Looking at upgrading etherpad servers to Xenial and realize that we
haven't "digitized" them yet. Lets get that in first so that the new
instances we boot can be etherpad01 and etherpad-dev01.
Change-Id: I700a1d018ab37436f11b2d883d79c82db7f1d6b1
We don't want to run ansible if we don't get a complete inventory from
our clouds. The reason for this is we cannot be sure that the ordering
of git servers, gerrit, and zuul or our serialized updates of afs
servers will work correctly if we have an incomplete inventory.
Instead we just want ansible to fail and try again in the future (we can
then debug why our clouds are not working).
From the ansible docs for any_unparsed_is_failed:
If 'true', it is a fatal error when any given inventory source
cannot be successfully parsed by any available inventory plugin;
otherwise, this situation only attracts a warning.
Additionally we tell openstack inventory plugin to report failures
rather than empty inventory so that the unparsed failures happen.
Change-Id: I9025776af4316fbdd2c910566883eb3a2530852a
There is an ERB parsing error for puppet 4, roll back while we figure it
out.
This reverts commit a156accb7d2799cb76e263c927f6bcfaac3c22b2.
Change-Id: I1d2cd468a8d1ebab2573b03250bf6c3ba1195798