160 Commits

Author SHA1 Message Date
Monty Taylor
fcab9a407a Update opendev git references in puppet modules
Not updating the gerrit git links thing, because that needs to
be a wider patch that updates the link syntax too.

Change-Id: I98013ba79e707540879e0cf2849a35c52f3371e8
2019-04-20 18:26:07 +00:00
Monty Taylor
c716240692
Clean up puppetmaster puppet config handled by ansible
We have a bunch of this handled now in ansible, so remove the old stuff.

Remove puppetmaster group management files. It's confusing for there to
be two files. Remove the old one.

Remove mqtt config. This isn't really a thing currently, and we're
eyeing running things from zuul anyway, so no need to port to ansible.

Change-Id: I8b64d21eadcc4a08bd5e5440fc5f756ae5bcd46b
2018-08-17 11:53:52 -05:00
Monty Taylor
245609bc95
Remove bridge from disabled and add puppet group
Instead of just having bridge be disabled, make a puppet group that it's
not a part of and switch the remote_puppet_else playbook to use that.

Change-Id: Ifb96ce483fc5675d095723bda70242a425bdc619
2018-08-15 08:43:23 -05:00
Colleen Murphy
4b5feca5cc Turn the future parser on for etherpad-dev
The puppet 4 tests are passing for etherpad_lite so let's try it out for
real.

Depends-on: https://review.openstack.org/590023

Change-Id: Ia91bd3950c9f48505a3024a16300091cf42d7f69
2018-08-09 21:34:23 +02:00
Zuul
4c71367d59 Merge "Add base playbooks and roles to bootstrap a new server" 2018-08-03 21:43:15 +00:00
Monty Taylor
0bb4232586 Add base playbooks and roles to bootstrap a new server
We want to launch a new bastion host to run ansible on. Because we're
working on the transition to ansible, it seems like being able to do
that without needing puppet would be nice. This gets user management,
base repo setup and whatnot installed. It doesn't remove them from the
existing puppet, nor does it change the way we're calling anything that
currently exists.

Add bridge.openstack.org to the disabled group so that we don't try to
run puppet on it.

Change-Id: I3165423753009c639d9d2e2ed7d9adbe70360932
2018-08-01 14:57:44 -07:00
Zuul
d148f5f8ef Merge "Turn on future parser for ask-staging.o.o" 2018-08-01 20:30:43 +00:00
Zuul
23a47d5602 Merge "Put bridge.openstack.org into disabled group" 2018-07-31 22:29:34 +00:00
Monty Taylor
50bf5e316c
Put bridge.openstack.org into disabled group
We don't want to run puppet on it, but the next patch might take
slightly longer to land.

Change-Id: I9daef72c2641f5451647eb0389790dee6c4746be
2018-07-31 16:46:53 -05:00
Colleen Murphy
7c7ae48781 Turn on future parser for ask-staging.o.o
Although a few issues have been uncovered for askbot on Xenial or in
testing scenarios with SSL parameters, those shouldn't affect the
production trusty nodes, so flip the switch for ask-staging.

Change-Id: I5603fae2ea7bb67d233939323fa38816cd5aa016
2018-07-31 14:09:45 -07:00
Colleen Murphy
7f4ae5e4e1 Turn the future parser on for codesearch.o.o
The hound module tests are passing on puppet 4[1] so let's flip the
switch.

[1] http://logs.openstack.org/33/582533/1/check/puppet-beaker-rspec-puppet-4-infra/1300a8f/job-output.txt.gz

Change-Id: I0b5b2272e3cd1b6076248b0c16fd9904bd1436d6
2018-07-24 09:12:28 +02:00
Colleen Murphy
4edd67c282 Turn on the future parser for groups.o.o
This patch adds groups.o.o to the futureparser ansible group to have
ansible set parser = future in its puppet.conf.

Change-Id: I5a59f5855c42372cd16682ea7cb859c0ed38fa1d
2018-07-24 09:12:03 +02:00
Colleen Murphy
e846b244ef Turn on the future parser for graphite.o.o
The beaker tests for puppet-graphite pass with puppet 4[1] so let's flip
the switch on the parser.

[1] http://logs.openstack.org/22/582022/1/check/puppet-beaker-rspec-puppet-4-infra/eba2276/job-output.txt.gz

Change-Id: Ib1954a1c62e75cd22319438adc1fb4992d705b9d
2018-07-24 09:12:03 +02:00
Colleen Murphy
07ab7b835c Turn on the future parser for groups-dev
This patch adds groups-dev to the futureparser ansible group to have
ansible set parser = future in its puppet.conf.

Change-Id: Ide789a7f5751714adb913ebc50e965f21f09bc48
Depends-On: https://review.openstack.org/584341
2018-07-23 21:46:11 +02:00
Colleen Murphy
089c63f1a7 Use the future parser for review-dev.o.o
This change will configure puppet.conf with parser = future to turn on
the future parser with puppet 3 on review-dev.openstack.org without
upgrading puppet.

Change-Id: I36833385b94ba37823abe59936ccc11a98f36f52
2018-07-19 19:41:11 +02:00
Jeremy Stanley
02e4eb0fa8 Finish switching to using new ansible inventory
Change I76b1099bf0cf3bfead17f96e456cdce87d0e8a49 altered the name of
the inventory script, so reflect that in the corresponding
subprocess call in launch-node.py and a comment in the
expand-groups.sh script.

Change-Id: I4c2c762716813b5d59dcc1b623f5988c8aa7d490
2018-06-18 18:27:34 +00:00
Zuul
eea6c0e468 Merge "This patch impliments the survey spec: I3c389596373b94459a32a4e540d514a2941acbb1" 2018-05-31 19:46:25 +00:00
Anita Kuno
06bbf62fc4 This patch impliments the survey spec:
I3c389596373b94459a32a4e540d514a2941acbb1

Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
Change-Id: Iad9e5bde37c467b9930c354f4d0b312b219d05b3
Story: 2000691
Task: 3164
2018-05-11 17:02:29 -04:00
Colleen Murphy
ac9a031e32 Stop managing infracloud things
Infracloud is sadly deceased. The upside is we can delete a lot of code
we don't need anymore. This patch removes infracloud nodes from
site.pp so that the puppet-apply test no longer bothers to validate
them, removes the infracloud modules from modules.env so that we don't
bother to install those modules in puppet-apply and puppet functional
tests, and removes the infracloud-specific data from the public hiera.
Additionally stop the puppetmaster from trying to run the infracloud
ansible playbook and finally remove the chocolate region from nodepool's
clouds.yaml (vanilla was already done).

This patch leaves the run_infracloud.sh script and the
infracloud-specific ansible playbooks as well as the infracloud
manifests in the openstack_project puppet module. It's possible those
tools could come in handy in the future if we ever have another
infracloud, and leaving those tools in place doesn't add confusion about
which hosts are actually active nor does it leave cruft that gets
unnecessarily tested.

Change-Id: Ic760cc55f8e17fa7f39f2dd0433f5560aa8e2d65
2018-04-24 09:13:40 +02:00
Paul Belanger
d3071deb03
Create review group for hieradata
Rather then creating per fqdn hiera entries for secrets, move to use a
group. This avoids the need to duplicate data.

Change-Id: I2208343b5281f70fc0850c0fe4e85038a53ed189
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-23 09:56:53 -04:00
Ian Wienand
a7a0947980 Match numbered mirror-update servers
The site.pp matching is already setting the group; update
the group match.

Change-Id: Ifa35be1cce8f5e923edc38238f083b53e44042e0
2018-03-15 11:53:22 +11:00
Paul Belanger
c1ed9e0684
Create review-dev for hieradata
Rather then creating per fqdn hiera entries for secrets, move to use a
group. This avoids the need to duplicate data.

Change-Id: I748314f52aeb6d288a1b133b3c20402c236dfb45
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-03-14 15:01:14 -04:00
Zuul
929053f4a3 Merge "Avoid generated-groups when creating generated-groups" 2018-01-23 16:00:09 +00:00
Jeremy Stanley
ff2e74e971 Add zuul-scheduler ansible group for zuul01
Add the zuul-scheduler ansible group for hosts in the inventory
matching zuul[0-9]+.openstack.org.

Change-Id: I5d54d623dcbb12dd2645481a1ee74c2c5512a814
2018-01-15 22:18:11 +00:00
Zuul
60c4c3eb2b Merge "Remove additional dead jenkins related code" 2017-12-20 01:07:18 +00:00
Ian Wienand
6fff5a4bad Avoid generated-groups when creating generated-groups
As explained in the comment, with the generated-groups file in the
inventory, as we regenerate the groups we end up finding old entries
that have actually disappeared and keep putting them back in.

Change-Id: I86463ecf516c38bb08d3d45e706a0da61a33efdf
2017-12-20 10:30:45 +11:00
Monty Taylor
fc4511d881
Remove additional dead jenkins related code
We don't need a clean workspaces playbook, nor do we need to do anything
with that during renames. We don't need to reference machines that don't
exist in ansible groups. The launcher ssh config is not used by
anything.

Change-Id: Id3e9cddb06b6e47b6f07d9a39086f3b054b46bde
2017-12-19 15:07:06 -06:00
James E. Blair
4ccf277850 Add an authoritative hidden master
This runs bind as a hidden master nameserver so we can do all the
keysigning there, and then use nsd (or bind) as public authoritative
slaves.

Change-Id: Ifb2ad109103051fa13c4af1c7be1ca0ae98bb1a1
2017-12-19 08:57:33 -08:00
James E. Blair
fe92742558 Create paste hiera group
Change-Id: I0a9fd98c7a95bdf7046c5a56bbbd0fab3f27412e
2017-12-12 14:33:23 -08:00
Ian Wienand
fbd37cf0cb status.o.o xenial upgrade
Convert puppet for status01.o.o which will be a xenial host.
status.o.o will be redirected.

Change-Id: Iec638a06e7578d6d8ad2bdc5daf2ef5abba6607f
2017-12-12 10:18:46 +11:00
Zuul
fbb89a6148 Merge "Create a grafana group" 2017-12-11 22:16:21 +00:00
Paul Belanger
64d71b1f23
Add eavesdrop into groups.txt
Now that we are using a numeric group, we need to add it to groups.txt
and update our private hieradata to use groups too.

Change-Id: I732d3698b3dfb591c2d6fa71f53e7a27f6143950
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-11 16:22:02 -05:00
James E. Blair
0e6b631233 Create a grafana group
Numeric hostnames don't match FQDN hiera groups, so we need to
create an explicit group.

Change-Id: Ie1e55b934b7d36cc1cb3434381660d7d27bbe979
2017-12-11 13:18:19 -08:00
Paul Belanger
6a0cebeda2
Remove zuulv2 long lived servers
We no longer need our proposal or signing nodes. These are now managed
directly in zuulv3 jobs (via nodepool).

Also noticed wheel mirror group was never deleted, so removing that
too.

Change-Id: Ibca89052b8d27093e17a33cb738fd3855538dca1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-11 14:17:46 -05:00
Ian Wienand
60b89d662e Remove ci-backup-rs-ord.openstack.org
Migrate backups to new backup01.ord.rax.ci.openstack.org

We decided to start fresh backups on the new server, so this is ready
to go.  I have performed an initial backup on each server so it has
accepted the host key of the new server and been tested (I also fixed
up review-dev.o.o, which was rebuilt but keys not updated ... todo:
add this to puppet, but since it changes so infrequently not high
priority).

Change-Id: I0872f9fcf4a334d32f632b3cb04801deefab4fd1
2017-11-15 09:28:55 +11:00
Monty Taylor
6e1a088899
Add nl*.openstack.org to the nodepool group in hiera
It needs to be in the nodepool group so that it will get updated hiera
information so that its clouds.yaml file can be correct.

Change-Id: Icd06ba6d67c6c6a6a78dbb3a22ec5c744fde47ee
2017-09-03 16:13:02 -05:00
Jenkins
3e31d01c91 Merge "Sanitize ansible results in mqtt callback plugin" 2017-08-02 18:16:00 +00:00
Paul Belanger
f87608d151 Add CA service to puppetmaster.o.o for zuul
We want to start encrypting our gearman traffic for zuulv3, as such
we'll need to bring online a CA service. The idea here, is we create a
new CA for each interconnecting service we want SSL certs for.

As an example /etc/zuul-ca will be used to generate SSL certs for our
gearman service.

Change-Id: I8c341559292c78d5428fe16837f28494a76e65db
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Co-Authored-By: Jeremy Stanley <fungi@yuggoth.org>
2017-06-16 18:27:59 +00:00
Paul Belanger
9af56a68af
Add zuul-executor to groups.txt
This is needed to copy private hieradata files to our zuul-executor
servers.

Change-Id: I85fe6a8c85ebed5662010571d3a0f9e46cd918c8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-08 11:14:18 -04:00
Jeremy Stanley
477b6093e0 Sanitize ansible results in mqtt callback plugin
According to https://github.com/ansible/ansible/issues/22505 the raw
raw results returned should be sanitized with the
CallbackBase._dump_results() method to ensure sensitive strings such
as those flagged by no_log are elided.

Change-Id: Iaebba820ffcb8628cf1e2373546e51ffc02deed6
2017-04-29 15:07:02 +00:00
Matthew Treinish
1ef6088bb6
Make playbook and task in topic singular
The topic names need to be consistent between event types to ensure
people can properly use wildcards to filter the messages on just the
things they care about. However playbook and task were inconsistent in
topics from between different methods in the callback plugin. This
commit fixes that by making sure we always use 'playbook' and 'task',
not 'playbooks' or 'tasks'.

Change-Id: I3e6240560ad562e8f41f7e314ef7a4b0b1178e32
2017-04-28 20:18:56 -04:00
Matthew Treinish
2bf2adb271
Cast the playbook uuid as a string
This commit fixes a missing cast for the playbook uuid so we can use it
in the topc. Without this we get a type error and nothing works.

Change-Id: Ib3a814bc93d9685d9b735a462c6ed56ad4370cfb
2017-04-28 19:31:33 -04:00
Matthew Treinish
3a8869a0f6
Enable ansible callback events on firehose
This commit adds the mqtt ansible callback plugin to the puppetmaster
config so that whenever we run ansible we'll emit events to the
firehose for that.

Change-Id: Id5f10705687c5bb9854d386efd7fed486172f745
2017-04-26 16:03:23 -04:00
Clark Boylan
bf1a15b4e6 Add translate ansible group
This was missed as part of the earlier translate upgrade change. With
switch to digitized host names we also want to group those hosts
together in hiera and ansible. Make that change here.

Change-Id: I6d25b35efbf0b43bd63a8ff9e217b68663575c75
2017-02-27 11:03:27 -08:00
Jeremy Stanley
2be925f8e9 Add HTTPS to developer and docs.openstack.org
Add X.509 certificates, certificate chains and private keys for
https://developer.openstack.org/ and https://docs.openstack.org/
separately using SNI (as the list grows we can consider condensing
these into a single cert using ServerAltNames later).

Change-Id: Ia365be3363b611e5ee3b6dceb38ec311456466ec
2017-01-25 23:32:10 +00:00
Clark Boylan
5700271dd5 Deploy simple ethercalc server
This is a simple first deployment of an ethercalc service. It does not
come with authenticated redis or redis backups. It will however have
working ssl.

Change-Id: I8c434a6bff42bce75e67fb37665d213f3cc018c8
Depends-On: Id10247211d9643e81bb1b6e8fb67377ba6de873a
2017-01-25 10:23:07 -08:00
Marton Kiss
42bc347adf Enable ask-staging.openstack.org puppet run
Backup from production was successfully restored, so we can restore
the services, reenable puppet run now.

Change-Id: I4f0a7452072aa24215b01d54e98ccc0712e53ad5
2017-01-02 15:40:43 +00:00
James E. Blair
f5d625ac6b Add nodepool hiera group
Also sort the groups.

Change-Id: I6c8718894c065549d5dfb37dae26a9c157c9cb09
2016-11-23 11:29:12 -08:00
Clark Boylan
2c2c3fd5f7 Do the other half of translate-dev grouping
We also need to make sure the group info is reflected in the ansible
groups.txt file. This new group which match the old and new servers. Can
change the * to + once the old server is gone.

Change-Id: I8da12d800e472c5bbd8245277269c4cf4774da14
2016-11-22 14:09:57 -08:00
Jenkins
fd854b90d8 Merge "Add cacti01 replacement node to Puppet" 2016-10-12 18:02:37 +00:00