Commit Graph

249 Commits

Author SHA1 Message Date
Jeremy Stanley
2fbf6d9e7a Stop managing OpenStackID servers
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.

OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.

Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
2021-08-31 19:53:13 +00:00
Clark Boylan
495bec8f2f Cleanup unused puppet modules from modules.env
This reduces the total number of git repos we need to manage as we
migrated away from puppet. Keeping this pruned is a good way of tracking
progress and should make the jobs more reliable until we can delete
them.

In this change remove modules that have moved to base server roles
like ntp and haveged. Also removed are ircbot management,
selinux (no more centos here), haproxy (this moved into ansible with
gitea), and lodgit modules.

Change-Id: I13e254541aed4ac657b696dd8e8b00c8340fe034
2021-08-04 16:14:24 -07:00
Clark Boylan
30a916ff94 Cleanup puppet things from zuul where we don't puppet anymore
This reduces the scope of our puppet related testing to things that
continue to use puppet. This is probably not strictly necessary but
helps keep us up to date with our TODO list.

Change-Id: I52bfff09ad0ddeabe7ad151bcf88c912f86a76ec
2021-05-21 17:03:08 -07:00
Ian Wienand
c49ece9204 Cleanup grafana.openstack.org
The opendev.org server is in production, cleanup the old puppet-based
host.

Change-Id: I6db3ce929226a23b96234b52ece8b17f4c6a326a
2020-10-29 07:59:42 +11:00
Monty Taylor
8c9b4af143 Stop cloning more puppet modules
Previous review pointed out some additional modules we probably
aren't using any longer.

Remove the openafs::client section from openstack_project::server
because we're doing this with ansible now.

Depends-On: https://review.opendev.org/733890
Change-Id: Ib5104da9cf7d53b77191f48ec185f5d667d51944
2020-06-05 12:09:30 -05:00
Monty Taylor
96364a11d9 Stop cloning a bunch of puppet modules we don't use
We've stopped using many of these, but we never got around to
removing them from lists.

Also, we should probably retire the repos.

Depends-On: https://review.opendev.org/717620
Depends-On: https://review.opendev.org/720527
Change-Id: I8e012c5bfa48d274dbd7f5484a9e75fee080cb5e
2020-06-05 08:42:47 -05:00
Monty Taylor
9abec21f8f Stop cloning drupal puppet modules
These were for groups.openstack.org which is no longer a thing.
We can retire puppet-drupal too.

Change-Id: I4a9ef3bf37545429ae7e1371be5806e26cef953e
2020-06-04 07:44:36 -05:00
Monty Taylor
9fd2135a46 Split eavesdrop into its own playbook
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.

Add the ability to override the keys for the zuul user.

Remove openstack_project::server, it doesn't do anything.

Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.

Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
2020-04-23 14:34:28 -05:00
Monty Taylor
5468f49254 Remove unused gerrit puppet things
We ain't using em.

Change-Id: I4ce9188a6b6a7e6a670e61bb17ab07e890faebcf
2020-04-19 10:59:25 -05:00
Monty Taylor
711295e918 Remove old etherpad.openstack.org
Once the DNS is swapped over to point at etherpad.opendev.org
we can delete the old stuff.

Change-Id: I626dd22b22a23619fcf460533336f1ddfec615d9
2020-04-19 10:58:46 -05:00
James E. Blair
42574b2b37 Run ZK from containers
Migration plan:
* add zk* to emergency
* copy data files on each node to a safe place for DR backup
* make a json data backup: zk-shell localhost:2181 --run-once 'mirror / json://!tmp!zookeeper-backup.json/'
* manually run a modified playbook to set up the docker infra without starting containers
* rolling restart; for each node:
  * stop zk
  * split data and log files and move them to new locations
  * remove zk packages
  * start zk containers
* remove from emergency; land this change.

Change-Id: Ic06c9cf9604402aa8eb4bb79238021c14c5d9563
2020-04-17 08:43:09 -07:00
Monty Taylor
43fdea4ac4 Remove bugday from status.o.o
It doesn't seem like this is used anymore. Let's remove it before
we update the rest of this, so that we don't have to, you know,
update abandoned things.

Change-Id: I1c3708021046a428da82eaa843961091915ba4af
2019-05-24 14:56:50 -05:00
Zuul
d968256e89 Merge "Update puppet-python to "fix" broken facts" 2019-05-14 23:38:48 +00:00
Monty Taylor
c6d129a108 Update some paths for opendev
There's a bunch in here. This is mostly big-ticket things and test
fixes. Also, change the README to rst - because why is it markdown?

Depends-On: https://review.opendev.org/654005
Change-Id: I21e5017011e1111b4d7a9e4bf0ea6b10f5dd8c1b
2019-04-20 09:31:14 -07:00
Ian Wienand
fb12d2620b Update grafana for new archive repo
Grafana has moved to a new package repo [1] and the apt-get update
step is failing on the current host.

The first version of puppet-grafana that has this update is 6.0.0;
this is two years later than the current version we're using so
... yeah.  It does not work with puppet3, so only run apply test with
puppet4.

It looks like upstream has moved from camptocamp/archive to
voxpupuli/archive so the comment is no longer required.

[1] https://grafana.com/blog/2019/01/05/moving-to-packages.grafana.com/

Change-Id: Ibab0ed6799563ba0f9674ef1ea575c6ac5d60341
2019-04-15 12:05:28 +10:00
Clark Boylan
f5fe78bb49 Update puppet-python to "fix" broken facts
The puppet-python module had some broken facter custom facts that they
removed in 1.9.5. After upgrading to puppet-4 we notice this now. Update
to 1.9.5 from 1.9.4 to fix the error.

Error from puppet log:

  Apr 11 00:15:30 afs01 puppet-user[5184]: (Facter) error while resolving custom fact "virtualenv_version": undefined method `[]' for nil:NilClass

Change-Id: I853ed9e212885b01944dd5dc9157f2abdd9dedc3
2019-04-11 09:50:20 -07:00
Zuul
3f8a94228e Merge "Bump puppet/staging to 1.0.1 to satisfy mysql module dep" 2019-03-06 20:07:39 +00:00
Clark Boylan
a3d7f61e05 Install latest puppet-vcsrepo
Puppet4 is a lot pickier about puppet module dependency resolution. On
some hosts with puppet4 we are getting:

  ModuleLoader: module 'project_config' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules

Running the suggested command we see that vcsrepo is not resolved
because the version of vcsrepo we install doesn't ahve a metadata.json.
Thankfully the HEAD of vcsrepo does have metadata.json which should
allow this to be resolved.

Depends-On: https://review.openstack.org/641161
Change-Id: I022afd8bab58588e31f3f8e806183933a1de4e6b
2019-03-05 15:46:27 -08:00
Clark Boylan
319e42a1c2 Bump puppet/staging to 1.0.1 to satisfy mysql module dep
Puppet 4 is a lot more particular about module deps and has notices that
our mysql module version depends on puppet/staging >=1.0.1 but we only
have 1.0.0 installed. Fix this dep issue by bumping to 1.0.1.

Change-Id: I36629821d8f8187dbffd614d3b263a9bc44ced6e
2019-03-05 15:07:33 -08:00
Zuul
dbb6ef7c0b Merge "Update puppetlabs-postgresql to 4.8.0" 2019-02-26 19:46:46 +00:00
Clark Boylan
b8b1fdde75 Nameservers are now managed with ansible
Remove the puppetry for managing nameservers as we now use ansible
configured name servers without puppet.

We will need to follow this up with deletion of the existing
ns*.openstack.org and adns1.openstack.org servers.

Change-Id: Id7ec8fa58c9e37ce94ec71e4562607914e5c3ea4
2019-01-08 16:16:49 -08:00
Ian Wienand
3059c72aa8 Update puppetlabs-postgresql to 4.8.0
This appears to be the first version that supports Xenial (16.04).
Choose this to be as close to what we have but no further than we
need.

Change-Id: Ibf383ec80f1b1a773874663b7db1f58e932c9eba
2018-11-21 22:49:04 +01:00
Ian Wienand
85297878f5 Retire bandersnatch mirroring
Bandersnatch mirroring has been disabled since
I88a838cb28fee3bd16b2b0a26e614ac5c2f23241 which is currently almost 6
months ago.  Since then we have been running a reverse caching proxy.

Although bandersnatch served us well, it seems pypi has become
impractical to mirror locally.  This is partially due to 2TB volume
limitations of OpenAFS and partially due to us not having a sane way
to filter large, frequently updating packages.  With the reverse proxy
working there are no plans to restore our local mirror.

Retire the references to it before we clean up the AFS volumes.

Change-Id: Ia23828328dd859bbf26f95735c1c2e99c573d10e
2018-11-16 10:28:11 +11:00
David Shrewsbury
6a8707a02d Enable unattended upgrades
Debian gets this from 'unattended-upgrades' and Red Hat family
gets it from yum-cron.

Change-Id: Ie00f6073ad1127b11a89e69816baf47011480dbf
2018-08-28 12:05:15 -04:00
James E. Blair
3d166f99f6 Add unbound role
Add it to the base playbook and add a testinfra test for it.

Change-Id: Id5098f33aac213e6add6f061684d0214dc99ab5b
2018-08-27 13:29:18 -07:00
David Shrewsbury
b3b698c6ff Add timezone role
Contains a handler to restart crond when tz is changed. Cron service
name differs across distros.

Removes the puppet-timezone usage.

Change-Id: I4e45d0e0ed37214ac491f373ff2d37750e720718
2018-08-27 07:34:28 -07:00
Zuul
8f8d0ac127 Merge "Remove references to long-gone odsreg.o.o service" 2018-08-13 16:53:51 +00:00
Colleen Murphy
46ebeeb4fb Remove references to stackalytics.o.o
stackalytics.openstack.org does not resolve and seems very dead. Remove
its node from site.pp and remove it from the docs to avoid confusion
about what servers we're really managing. We can always add it back when
the time comes to try again.

Change-Id: I733130ebe97ae7e06ca57b3c8e3a8708fcfa069c
2018-08-07 22:12:43 +02:00
Jeremy Stanley
30f3039f1e Remove references to long-gone odsreg.o.o service
The odsreg.openstack.org server was removed from service 2018-01-11,
and should also be cleared from our global site manifest. It was the
only thing utilizing the puppet-odsreg module, so we can take that
out of the modules list as well (that will be retired separately).

Change-Id: Iadfddb3bf57428b928cacaaa672e24c4a1e92058
2018-08-07 19:52:13 +00:00
Ian Wienand
44df1b5fd5 Update puppet-grafana
This module has been deprecated for the voxpupuli version.  2.6.1 is
the last release with Puppet 3 support.  It seems to mostly be a fork,
so I think it is compatible.  The repo is hard-coded which is why I
think it's worth updating.

Change-Id: I3ca7c04a4e6122625e716682f8e0d4f9b62dbfc0
2018-06-06 14:50:55 +10:00
Colleen Murphy
ac9a031e32 Stop managing infracloud things
Infracloud is sadly deceased. The upside is we can delete a lot of code
we don't need anymore. This patch removes infracloud nodes from
site.pp so that the puppet-apply test no longer bothers to validate
them, removes the infracloud modules from modules.env so that we don't
bother to install those modules in puppet-apply and puppet functional
tests, and removes the infracloud-specific data from the public hiera.
Additionally stop the puppetmaster from trying to run the infracloud
ansible playbook and finally remove the chocolate region from nodepool's
clouds.yaml (vanilla was already done).

This patch leaves the run_infracloud.sh script and the
infracloud-specific ansible playbooks as well as the infracloud
manifests in the openstack_project puppet module. It's possible those
tools could come in handy in the future if we ever have another
infracloud, and leaving those tools in place doesn't add confusion about
which hosts are actually active nor does it leave cruft that gets
unnecessarily tested.

Change-Id: Ic760cc55f8e17fa7f39f2dd0433f5560aa8e2d65
2018-04-24 09:13:40 +02:00
Zuul
fb42517b81 Merge "Update puppet-nodejs to 2.3.0" 2018-04-09 16:12:09 +00:00
James E. Blair
4ccf277850 Add an authoritative hidden master
This runs bind as a hidden master nameserver so we can do all the
keysigning there, and then use nsd (or bind) as public authoritative
slaves.

Change-Id: Ifb2ad109103051fa13c4af1c7be1ca0ae98bb1a1
2017-12-19 08:57:33 -08:00
Zuul
154876b422 Merge "Add dns servers" 2017-12-15 14:51:47 +00:00
James E. Blair
83ba1311df Add dns servers
Change-Id: I32b0d846cbbaad5755d3d1c47d303b7cdf34f749
Depends-On: Ic92726dc341af5802ad803d239bd547ef5068043
Story: 2001382
Task: 6090
2017-12-14 17:04:54 -08:00
James E. Blair
0a57d5156c Add dnsquery puppet module
Change-Id: I29d36cc527351e3e6d2ee2dc1919988379b8db3a
2017-12-14 11:16:14 -08:00
Paul Belanger
fa7c94f150
Bump puppetlabs-apache to 1.11.1
This is currently the last 1.x release that still support puppet 3, we
could make the jobs to puppet 4, but for now we just need a new enough
module for xenial support.

Also, seems puppetlabs-apache is only uesd by cacti.o.o.

Change-Id: I128a0d8d851311b77592d98ded5891d71dce2031
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-13 16:44:15 -05:00
Zuul
8ac334eca9 Merge "Retire apps site remains" 2017-12-13 18:36:07 +00:00
Ian Wienand
50efa4ada1 Update puppet-nodejs to 2.3.0
This is the lastest version of puppet-nodejs that both supports Xenial
and puppet 3 ... after this it starts using puppet 4 syntax.  Update
because we need it for Xenial.  (note they also changed to using a "v"
prefix)

Change-Id: Ia7966fb9578d0d79f3a7f9480e3a956555737dc8
2017-12-12 10:06:09 +11:00
Paul Belanger
a3a2a84d8a
Remove puppetdb / puppetboard server
Currently puppetdb and puppetboard have been broken for some time (+1
year) and with ubuntu precise becoming EOL it is prime for deleting.

This leaves openstack-infra with a gap in reporting for non-root
users.  As such, as proposal is in the works to maybe use ARA.

Change-Id: Ifc73a2dba3b37ebe790a29c0daa948d6bad0aa33
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-12-11 10:11:11 -05:00
Andreas Jaeger
233b4b0257 Retire apps site remains
The app catalog was retired earlier this year, see
http://lists.openstack.org/pipermail/openstack-operators/2017-July/013965.html

The site is retired as well.

We can remove now any remains about these.

Change-Id: I6bb5740e622cbf214fd6d94847a5cf93cc01f1f8
Depends-On: I638cc127d3588d757a4da01984a2bd292a84413c
2017-12-10 20:24:20 +01:00
Jeremy Stanley
5baefe21af Install ptgbot on eavesdrop.o.o
This installs the ptgbot Puppet module so it will run from the
eavesdrop.openstack.org server and generate its Web content there.
Include some rudimentary operational documentation.

Change-Id: I92ddbbb683dede2c325f70267bd5e26884a35c01
Depends-On: Idb1fc5273b67ab88e1c78578275969b04c781c7a
2017-07-06 21:03:48 +00:00
Monty Taylor
ce8082cdb0
Use mitaka-eol tag for infra cloud puppet modules
infra-cloud still uses mitaka, but the stable/mitaka branch is not
around, so we can't use that as a git ref.

Change-Id: Ib7456376111a630c2e99e6d5a45bd39e486e60ae
2017-07-05 12:13:52 -05:00
Jenkins
60a54e7b09 Merge "Bump puppetlabs-apt and puppetlabs-stdlib" 2017-06-14 19:16:04 +00:00
Colleen Murphy
f8d5f88de1 Bump puppetlabs-mysql to 3.11.0
The mysql module was hardcoding 'upstart' as the mysql service provider,
which is wrong on xenial. This was fixed in 3.7.0[1] but let's go ahead
and bump it up to the latest version, which is still not a major version
bump.

[1] https://github.com/puppetlabs/puppetlabs-mysql/blob/master/CHANGELOG.md#supported-release-370

Change-Id: If25da754ec107b8dba2d9343f16b54b99cc88e01
2017-06-10 22:45:11 +02:00
Colleen Murphy
aedfe923c6 Bump puppetlabs-apt and puppetlabs-stdlib
puppetlabs-apt 2.3.0 contains a bugfix to make apt::ppa idempotent[1],
which will help us fix our puppet-diskimage_builder beaker tests.
puppetlabs-apt 2.4.0 is the latest puppet 3 compatible version
available. Unfortunately it also introduced useage of the deprecated()
function, which wasn't introduced in stdlib until stdlib 4.13.0[2]. This
patch goes ahead and bumps stdlib all the way to the latest major
release which is, surprisingly, not a major version bump for us.

[1] https://github.com/puppetlabs/puppetlabs-apt/blob/master/CHANGELOG.md#supported-release-230
[2] https://github.com/puppetlabs/puppetlabs-stdlib/blob/master/CHANGELOG.md#supported-release-4130

Change-Id: I98dfc07c955e1fe90905b619d16129544c27e861
2017-06-10 16:10:20 +02:00
Matthew Treinish
eb4c825215
Add mqtt_statsd to firehose config
To get metrics on MQTT usage into graphite/grafana this commit adds
running mqtt_statsd on firehose

Change-Id: I90bb2c4fc7e409e9af24ca7cec7ad9d7926739e9
Depends-On: I28058bf6eac2354e3ceba0011464509ed6bdd869
2017-04-11 16:44:32 -04:00
Jenkins
f094b61df5 Merge "Update biemond-wildfly to version 1.2.4" 2017-01-26 22:24:42 +00:00
Clark Boylan
5700271dd5 Deploy simple ethercalc server
This is a simple first deployment of an ethercalc service. It does not
come with authenticated redis or redis backups. It will however have
working ssl.

Change-Id: I8c434a6bff42bce75e67fb37665d213f3cc018c8
Depends-On: Id10247211d9643e81bb1b6e8fb67377ba6de873a
2017-01-25 10:23:07 -08:00
Clark Boylan
43d8101b43 Update biemond-wildfly to version 1.2.4
We were using version 0.4.0 which supported Wildfly 9 just fine (with
the systemd sysv support hack), but now we need wildfly 10 on the
translate-dev server. Unfortunately 0.4.0 does not support wildfly 10
properly because wildfly 10 removed its built in init scripts. This
newer puppet module will install systemd unit files that should work
fine.

Note that even though the new version of the puppet module supports
wildfly 8, 9, and 10 it does not appear to do so on all ubuntu releases.
Wildfly 10 requires a current enough release to have proper systemd.
Older wildfly versions shoudl work on older ubuntu releases via the
built in init scripts.

Depends-On: I39453506821ff7073290a4e5696eda8575b6cb06
Change-Id: Iac6abcc4644276b1fc3117737e1dd4aa1be87785
2017-01-20 13:23:03 -08:00